├── .github └── workflows │ └── buildsite.yml ├── .gitignore ├── README.md ├── bin └── build ├── cpanfile ├── docs ├── future.md ├── motivation.md ├── others.md ├── process.md └── template.md ├── images ├── flowchart.graffle ├── flowchart.md └── flowchart.png ├── in ├── index.tt └── style.css ├── lib └── PPC.pm ├── ppcs ├── ppc0001-n-at-a-time-for.md ├── ppc0003-np-warning.md ├── ppc0004-defer-block.md ├── ppc0005-everything-slices.md ├── ppc0006-load-module.md ├── ppc0007-source-encoding.md ├── ppc0008-sv-boolean-type.md ├── ppc0009-builtin-namespace.md ├── ppc0011-slurp-argument.md ├── ppc0012-configure-no-taint.md ├── ppc0013-overload-join-and-substr.md ├── ppc0014-english-aliases.md ├── ppc0015-drop-old-package-separator.md ├── ppc0016-indexed-builtin.md ├── ppc0017-original-build-type.md ├── ppc0018-module-true.md ├── ppc0019-qt-string.md ├── ppc0020-lexical-export.md ├── ppc0021-optional-chaining-operator.md ├── ppc0022-metaprogramming.md ├── ppc0023-map-grep-with-topic.md ├── ppc0024-signature-named-parameters.md ├── ppc0025-perl-version.md ├── ppc0026-enhanced-regex-xx.md ├── ppc0027-any-and-all.md ├── ppc0028-custom-prefix-postfix-operators.md ├── ppc0029-attributes-v2.md ├── ppc0030-undef-aware-equality.md ├── ppc0031-metaoperator-flags.md └── ppc0033-ampersand-method-calls.md └── ttlib └── page.tt /.github/workflows/buildsite.yml: -------------------------------------------------------------------------------- 1 | name: Generate web page 2 | 3 | on: 4 | push: 5 | branches: 'main' 6 | workflow_dispatch: 7 | 8 | jobs: 9 | build: 10 | runs-on: ubuntu-latest 11 | container: perl:latest 12 | 13 | steps: 14 | - name: Perl version 15 | run: perl -v 16 | 17 | - name: Checkout 18 | uses: actions/checkout@v4 19 | 20 | - name: Install pandoc and cpanm 21 | run: apt-get update && apt-get install -y pandoc cpanminus 22 | 23 | - name: Install modules 24 | run: | 25 | cpanm --installdeps --notest . 26 | 27 | - name: Get repo name into environment 28 | run: | 29 | echo "REPO_NAME=${GITHUB_REPOSITORY#$GITHUB_REPOSITORY_OWNER/}" >> $GITHUB_ENV 30 | 31 | - name: Create pages 32 | env: 33 | PERL5LIB: lib 34 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 35 | run: | 36 | mkdir -p web 37 | perl bin/build $REPO_NAME 38 | 39 | - name: Update pages artifact 40 | uses: actions/upload-pages-artifact@v3 41 | with: 42 | path: web/ 43 | 44 | deploy: 45 | needs: build 46 | permissions: 47 | pages: write 48 | id-token: write 49 | environment: 50 | name: github-pages 51 | url: ${{ steps.deployment.outputs.page_url }} 52 | runs-on: ubuntu-latest 53 | steps: 54 | - name: Deploy to GitHub Pages 55 | id: deployment 56 | uses: actions/deploy-pages@v4 57 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | web/ 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | This repository is for *Proposed Perl Changes* - proposals to change the Perl language. 2 | 3 | Right now, we're [trialling the process](docs/process.md). If you would like to submit a feature request, please [email an *elevator pitch*](mailto:perl5-porters@perl.org) - a short message with 4 paragraphs: 4 | 5 | 1. Here is a problem 6 | 2. Here is the syntax that I'm proposing 7 | 3. Here are the benefits of this 8 | 4. Here are potential problems 9 | 10 | and if a "paragraph" is 1 sentence, great. 11 | 12 | That will be enough to make it obvious whether the idea is 13 | 14 | 0) actually a **bug** - a change we'd also consider back porting to maintenance releases (so should be a opened as [*Report a Perl 5 bug*](https://github.com/Perl/perl5/issues/new/choose)) 15 | 0) worth drafting an PPC for 16 | 0) better on CPAN first 17 | 0) "nothing stops you putting it on CPAN, but it doesn't seem viable" 18 | 19 | You don't need to subscribe to the list to send an idea (or see updates). By keeping all discussion there during the trial, we can see if the process works as hoped, and fix the parts that don't. 20 | 21 | Please **don't** submit ideas as *issues* or *PRs* on this repository. (We can disable issues, but not PRs). Please follow the instructions above. 22 | 23 | These files describe the process we are trialling 24 | 25 | * [motivation.md](docs/motivation.md) - why do we want to do something 26 | * [process.md](docs/process.md) - the initial version of the process 27 | * [template.md](docs/template.md) - the PPC template 28 | * [future.md](docs/future.md) - how we see the process evolving 29 | * [others.md](docs/others.md) - what others do (or did), and what we can learn from them 30 | 31 | ## The Tracker 32 | 33 | The status of proposals is tracked in "[the PPC 34 | Tracker](https://docs.google.com/spreadsheets/d/1hVOS7ePuLbVkYcf5S-e_eAodj4izm9Cj7AVs25HvngI)", 35 | a Google Sheets spreadsheet. Keep up to date there! 36 | -------------------------------------------------------------------------------- /bin/build: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | use v5.40; 4 | use JSON; 5 | use File::Copy; 6 | use Template; 7 | use Template::Provider::Pandoc; 8 | 9 | use PPC; 10 | 11 | my @ppcs; 12 | 13 | my $outpath = './web'; 14 | my $template_path = [ './ppcs', './docs', './in', './ttlib' ]; 15 | 16 | my $base = shift || $outpath; 17 | $base =~ s/^\.//; 18 | $base = "/$base" if $base !~ m|^/|; 19 | $base = "$base/" if $base !~ m|/$|; 20 | 21 | my $provider = Template::Provider::Pandoc->new({ 22 | INCLUDE_PATH => $template_path, 23 | }); 24 | 25 | my $tt = Template->new({ 26 | LOAD_TEMPLATES => [ $provider ], 27 | INCLUDE_PATH => $template_path, 28 | OUTPUT_PATH => $outpath, 29 | RELATIVE => 1, 30 | WRAPPER => 'page.tt', 31 | VARIABLES => { 32 | base => $base, 33 | } 34 | }); 35 | 36 | for () { 37 | my $ppc = PPC->new_from_file($_); 38 | push @ppcs, $ppc; 39 | 40 | $tt->process($ppc->in_path, {}, $ppc->out_path) 41 | or warn $tt->error; 42 | } 43 | 44 | my $vars = { 45 | ppcs => \@ppcs, 46 | }; 47 | 48 | $tt->process('index.tt', $vars, 'index.html') 49 | or die $tt->error; 50 | 51 | for () { 52 | s|^docs/||; 53 | my $out = s|\.md|/index.html|r; 54 | 55 | $tt->process($_, {}, $out) 56 | or die $tt->error; 57 | } 58 | 59 | mkdir 'web/images'; 60 | for () { 61 | copy $_, "web/$_"; 62 | } 63 | 64 | if (-f 'in/style.css') { 65 | copy 'in/style.css', 'web/style.css'; 66 | } 67 | 68 | if (-f 'CNAME') { 69 | copy 'CNAME', "web/CNAME"; 70 | } 71 | 72 | my $json = JSON->new->pretty->canonical->encode([ 73 | map { $_->as_data } @ppcs 74 | ]); 75 | 76 | open my $json_fh, '>', 'web/ppcs.json' or die $!; 77 | 78 | print $json_fh $json; 79 | 80 | -------------------------------------------------------------------------------- /cpanfile: -------------------------------------------------------------------------------- 1 | requires 'File::Copy'; 2 | requires 'JSON'; 3 | requires 'Template'; 4 | requires 'Template::Provider::Pandoc'; 5 | -------------------------------------------------------------------------------- /docs/future.md: -------------------------------------------------------------------------------- 1 | # How do we scale this up so that it doesn't need so much central attention? 2 | 3 | The initial process assumes that 4 | 5 | * many people want to submit ideas 6 | * most of them don't understand what it takes to get an idea implemented 7 | * there are a lot of duplicate ideas 8 | * we don't have a clear way for them to know "what ideas are viable" 9 | * we don't know if the process will work 10 | 11 | so there's a deliberately a lot of hand holding and centralisation, so that we can steer folks in the right direction, or say no early, before they do too much work. 12 | 13 | ## If the process works, I'd like to progress to 14 | 15 | * people submit draft PPCs, not just ideas 16 | * they've actually consulted other people *before* submitting 17 | * the person submitting the PPC knows that it they don't help, it dies 18 | * there's a clearer set of requirements needed to progress 19 | 20 | ## What does the Author do/Where do PPCs live? 21 | 22 | The Author is the champion for the PPC. Their motivation and enthusiasm to have the feature successfully implemented and shipped drives the process. Their task is to eliminate each "lowest hanging excuse" in turn to get to shipping code. They are responsible for 23 | 24 | * seeking input 25 | * shepherding discussion to some sort of consensus (or appealing to the PSC to resolve an impasse) 26 | * ensuring all sections in the PPC are complete (working with implementers and others as necessary) 27 | 28 | PPCs live in version control. Anyone can create an "Draft" PPC, self-assign it an ID, and start to flesh it out. "Exploratory" status gets you an official PPC ID, and from that point onward the PPC and history is mirrored/merged into the official repository. 29 | 30 | To make this workable, the PPC should be in a source code repository that the *Author* can edit directly (GitHub, GitLab, Bitbucket, self-hosted git, etc). The minimal workable requirements are that 31 | 32 | 1. it gives a well-known stable URL for the rendered current version 33 | 2. it can tag (or identify) specific previous revisions 34 | 3. history can be mirrored into the official repo, and merged on status change 35 | 4. discussion can be archived once PPC is "Accepted" and "Implemented" 36 | 37 | Hence GitHub PRs might not be the best forum for discussion, unless they can be archived. Basically we want to avoid the situation were we have a feature live, and some third party can delete the historical discussion related to it. 38 | 39 | As we get better at this, I think that the status transitions should aim for these minimum requirements 40 | 41 | 42 | ## Draft 43 | 44 | * *Author* self-assigns PPC ID 45 | * *Author* seeks input/feedback 46 | 47 | ## Exploratory 48 | 49 | * MUST have *Sponsor* (on the core team, or PSC can delegate externally) 50 | * MUST have (at least minimal) Motivation, Rational and Examples 51 | * SHOULD have (at least a minimal) Specification 52 | * Gets an official PPC ID 53 | 54 | Means "we think this idea is worth exploring" 55 | 56 | ## Provisional 57 | 58 | * MUST have viably complete Motivation 59 | 60 | Means "we think this idea is worth implementing" 61 | 62 | ## Accepted 63 | 64 | * MUST have viably complete Motivation, Rational, Specification, Examples 65 | * MUST have Prototype Implementation/Proof Of Concept/Specific Plan 66 | 67 | Means "we think this plan looks viable" 68 | 69 | ## Implemented 70 | 71 | * MUST have no Open Issues 72 | * MUST actually have an implementation! 73 | 74 | Means "it's good to merge - we think we can support it in the future" 75 | 76 | ## Shipped 77 | 78 | * In a stable release, subject to "experimental features" process 79 | 80 | ## Stable 81 | 82 | * In a stable release, would need to follow "deprecation" process to remove 83 | 84 | 85 | 86 | where (at least) "Provisional" and "Shipped" can be skipped. 87 | 88 | ## PPC IDs, and how to self-assign 89 | 90 | * Official PPC IDs are 4 digits 91 | * Self-assigned IDs should be `CPANID-#### or` `githubid-####` 92 | 93 | These are the obvious likely popular two, and done case sensitively will not clash. These two should be sufficient for a workable globally unique system. 94 | -------------------------------------------------------------------------------- /docs/motivation.md: -------------------------------------------------------------------------------- 1 | # Top line 2 | 3 | 80% of our feature requests are for changes to the language, and most seem to come from folks with no experience of core development. Whereas over the past 5 years most work on `toke.c` and `perly.y` is from just 5 people 4 | 5 | Karl, LeoNerd, Tony, Zefram, Dave 6 | 7 | (with the first 3 far more prominent) 8 | 9 | There's a massive disparity between how many ideas we have and how many people we have to even **mentor** others, let alone implement things. 10 | 11 | To get out of this hole, we need an approach that acknowledges this and emphasises scaling up and out with what we have, instead of pretending that we're short on ideas and long on under-used talent. 12 | 13 | # Meaning 14 | 15 | Sure, we do not want to rule out "unsolicited" ideas, but we **have** to assume that these were unlikely to go far unless someone else with the right skills buys into them. 16 | 17 | I really don't want to just create a "roadmap" of "aspirations" that blocks for years because no-one implements it. "Yes, we're going to add desugaring assignment..." 18 | 19 | # So I want it to be clear 20 | 21 | * How the person with the idea can contribute and help get a lot done 22 | * How much work there really is 23 | * That **your** idea needs to be applicable to other people ... 24 | * ... and it needs at least 1 more person prepared to help implement it 25 | 26 | # I want to avoid 27 | 28 | * [The Perl 6 RFC failure](https://www.perl.com/pub/2000/11/perl6rfc.html/) of "no-one knew how to implement it" 29 | * Throw an idea over the fence and assume that it will be picked up 30 | * "Quantity over quality" of ideas 31 | * A process that is too verbose to read 32 | * Different people (repeating) the same feedback/what-if questions, because the answers to these are not easy to locate, resulting in noise and makework. 33 | 34 | 35 | # Bugs, optimisations, build improvements etc 36 | 37 | * can objectively and uncontroversially be triaged as accepted or rejected 38 | * rarely conflict with each other 39 | * once completed and tested can generally be closed and forgotten 40 | 41 | # Feature requests are 42 | 43 | * subjective 44 | * involve trade offs that different people value differently 45 | * have more states that "open" and "closed" 46 | * and even rejected ideas should not be forgotten, because the reasoning for rejecting them is often interesting. 47 | 48 | It's not helpful to place both in the same "queue" and process them as variants of the same thing, because they are not. They don't even need to be in the same repository - you don't need the possible future history of Perl in order to build it from source, or use it to write working programs. 49 | 50 | We can strive to reach 0 bugs. 51 | 52 | Unlike bugs, the functionality of released Perl doesn't suffer if we have infinite open features. We gain nothing from striving to reach 0 open feature requests. 53 | 54 | # I'm really keen to 55 | 56 | 1. Split the bug queue from the **idea** queue 57 | 2. Distinguish between "someone's idea" and "idea we like and think is viable" 58 | 3. Be clear that there are no magic coding fairies (it should be the exception rather than the rule that we accept ideas without an idea of who is going to do it) 59 | 4. Be clear that there is a bunch of project management stuff that folks **can** help with to champion their feature - that's how they help us scale. "oh, but I can't hack C so clearly I can't help here" is a misconception 60 | 5. Give equal prominence to ideas that didn't make it 61 | 6. And of assessments of why 62 | 63 | 64 | # What are we trying to achieve 65 | 66 | * A structured way for anyone to contribute workable ideas to improve Perl 67 | * Permit folks without deep technical skills to be maximally helpful 68 | * Spread workload to offload as much as possible from the few people who can actually wrangle the internals 69 | * Distribute and decentralise where possible 70 | * Record decisions taken along the way 71 | * Create a record of what works/what doesn't to improve future contribution 72 | 73 | Basically the *MVP* - Minimum Viable Process. 74 | -------------------------------------------------------------------------------- /docs/others.md: -------------------------------------------------------------------------------- 1 | # What can we learn from others. 2 | 3 | This is a process for features, not for bug fixes. Obviously, it's 4 | unclear where the line is, and do we need it for trivial features? 5 | TCL's view was: 6 | 7 | > We divide projects into two general categories: bug fixes and feature changes. In general, if a project requires manual entries to be updated then it is a feature change; when in doubt, a project is a feature change 8 | 9 | If we don't learn from the mistakes documented in https://www.perl.com/pub/2000/11/perl6rfc.html/ we're doomed to repeat them. 10 | 11 | ## So what can we learn from others? 12 | 13 | "Python Enhancement Proposals" and "TCL Improvement Proposals" diverge from the same source in Aug 2000. I assume that PEPs were the original, and TIPs the fork because I can't find a common third source that both share. 14 | 15 | Hence these both predate Distributed Version Control Systems, easy efficient reliable free web hosting, and the common use of code review (or at least effective tools to do this remotely and in different timezones). 16 | 17 | TIPs seem to assume a small core team who are equally competent (and maybe not optimised for bad ideas or inexperienced contributors), whilst PEPs assume a Benevolent Dictator For Life (ie Guido) and then are structured to delegate work from him. PEPs seem to rely on a lot of work being done/doable by PEP editors. 18 | 19 | The PEP process has evolved (both before and after Guido's retirement). TIPs resolutely have not, and still mention SourceForge despite TCL now using Fossil. The Python Core Team is about 100, and the PEP process in now implemented in terms of of their steering committee or a delegated person. The TIP process only involves the core team, and the approvals process is "TYANNOTT: Two Yesses And No No's Or Two Thirds" (two approvals and no objections, or a full vote is needed) 20 | 21 | PHP uses RFCs tracked in wiki, and then voting by core team of about 40. 22 | 23 | PEPs must have a sponsor - this is someone on the core team. 24 | TIPs must have two people on the core team in favour. 25 | 26 | * https://wiki.php.net/rfc/howto 27 | * https://www.python.org/dev/peps/pep-0001/ 28 | * https://core.tcl-lang.org/tips/doc/trunk/tip/0.md 29 | * https://core.tcl-lang.org/tips/doc/trunk/tip/2.md 30 | 31 | PEPs and TIPs are both used both for code improvements and process improvements. This conflation seems awkward - it requires a special status ("Active") for process related PEPs/TIPs, and many of the sections needed for feature changes are irrelevant for process changes. It also means that the *official* document for the process itself is the change document for the process, not the procedure document. I think this is a bad idea and we should not imitate it. PHP's RFCs only cover code. I like this better, but I think we can "borrow" more of the detail from the Python and TCL approach than from PHP's. 32 | -------------------------------------------------------------------------------- /docs/process.md: -------------------------------------------------------------------------------- 1 | # Bootstrapping a PPC process 2 | 3 | PPC is the acronym for "Perl Proposed Change". 4 | 5 | 80% of our feature requests are for changes to the language. 6 | 7 | In the past 5 years most work on the parser and tokeniser came from just 5 people. All of them are busy, and none of them are employed by "Perl 5 Porters". 8 | 9 | We can't change this mismatch quickly - the reality is that even good ideas might stall for lack of anyone to implement them, and even where folks are prepared to help implement their ideas, we will find it hard to mentor many simultaneously. 10 | 11 | Hence we need a process that 12 | 13 | * acknowledges this 14 | * emphasises scaling what we have 15 | * enables the originator of an idea to help us as much as possible 16 | * summarises and records discussion and decisions, to iteratively improve 17 | * prioritises proposals, to optimise the value we get from contributors 18 | * clarifies who is meant to be pushing work forward at any given time 19 | 20 | 21 | We'd like to record proposals to improve the language and their status as "Request For Comment" documents in their own repository under the Perl organisation on GitHub. 22 | 23 | 24 | We have a [template](./template.md) for what an completed implemented PPC should end up as, but if all you have is an idea - don't worry, we'll help you get there. We're still figuring this process out, so for now we're doing it as mail messages sent to p5p, not as "pull requests" to the PPC repository (or "issues" on the source repository). This way we can see if the process works as hoped, and fix the parts that don't. 25 | 26 | 27 | ## What makes a good idea? 28 | 29 | Strictly speaking, as Perl is a Turing complete language, no changes to Perl are **necessary** because any task that can be implemented at all can be implemented with or without a proposed change. Arguing against a new feature because it is already possible is tautological. New features are suggested because they might make the language *better*. Better is subjective - we can't avoid different people weighing trade offs differently. Perl tries to make *easy things easy and hard things possible*, and is fine with *There Is More Than One Way To Do It*. 30 | 31 | All changes have costs and benefits. Benefits of a new approach could be that it 32 | 33 | * is less verbose than existing syntax 34 | * has fewer ways to make mistakes 35 | * is more efficient internally 36 | * opens up new ways to express problems 37 | 38 | Costs are 39 | 40 | * yet another way to do it - Perl becomes incrementally harder to learn, remember and read 41 | * existing tooling has to adapt to cope 42 | * linearly more implementation to maintain 43 | * exponentially more combinations of features to define and debug 44 | 45 | Not every good idea belongs in the Perl core. Some are better implemented on CPAN. For some ideas, the PPC process is overkill. And the other way - for some issues or PRs, the reviewer is going to realise that it's more complex than it seemed, and needs to become a PPC. 46 | 47 | ## The Process 48 | 49 | ![a flowchart of the process described below](../images/flowchart.png) 50 | 51 | 52 | ### Pre-PPC 53 | 54 | The PPC process starts with a formal proposal to add or change a language feature in Perl. But you, the prospective author of a PPC, shouldn't start by writing that formal proposal. Start by posting to p5p that you have an idea. Explain what problem you're solving, how you think you can solve it, and what prior art you looked at. Be clear and concise. Make it easy for the rest of the list to see what you're suggesting without reading an enormous wall of text, but don't lose so much detail as to be meaningless. 55 | 56 | You are taking the temperature of the list. If there is a great outcry that this is a bad idea, or has been tried before, or was explicitly rejected before, you should probably stop. No hard feelings! 57 | 58 | Otherwise, you're ready to move on to the next step. You should post a follow-up, requesting that the PSC approve producing a draft. If they do, move on to "Draft Proposal" below. If not, they'll provide more feedback like "this is definitely impossible" or "you need to provide more information" or so on. 59 | 60 | During this "Pre-PPC" phase, your proposal isn't in the PPC tracker. It's not a PPC yet! 61 | 62 | During this phase, you (the proposer) are responsible for moving things forward. If you contact the PSC for approval to file a draft, and the PSC does not respond, it's you who should be keeping track of that. 63 | 64 | ### Draft Proposal 65 | 66 | The PSC has agreed that you should write a formal draft proposal. You get the [template document](./template.md) and fill it out. You take its advice, thinking hard about what goes in each section. Then you post it to p5p as an email with the subject "PROPOSAL: my great idea". Members of the list will reply with more questions and suggested amendments. You should read them and amend the proposal to clarify your ideas or react to valid criticism. 67 | 68 | During this phase, you (the proposer) are responsible for moving things 69 | forward. 70 | 71 | When you think your idea has been sufficiently scrutinized, and you have gotten all the feedback you're going to benefit from, post to perl5-porters, requesting the PSC accept the draft. In reply, they will either request more discussion, reject the proposal, or enter it into the tracker with the status **Exploratory**. 72 | 73 | ### Exploratory Status 74 | 75 | When a formal draft has been discussed sufficiently, submitted to the PSC, and is under consideration, it is entered into the proposal tracker, with the status **Exploratory**. The PSC (or other deputized folk) will begin the process of vetting the idea. They will review discussion about the idea, they will produce a list of questions not yet answered, and they will press existing core team members (or other experts) for input. 76 | 77 | The PSC (or deputies) will eventually either: 78 | * move the document to **Implementing** status (see below) because they believe it is ready for implementation 79 | * move the document to **Rejected** status because they believe it should not be implemented. This may come with advice on how to formulate an alternate proposal that has more chance of being accepted. This isn't used for "it needs some edits", but for "it's fundamentally deeply flawed." 80 | * move the document to the **Expired** status because the original proposer (or other vital personnel) are not responsive 81 | 82 | During this phase, the PSC is responsible for moving the proposal forward. 83 | 84 | ### Implementing 85 | 86 | When a proposal is accepted, the PSC is stating that they'd like to see the idea implemented, and that they are likely to merge an implementation if it doesn't bring with it any unwelcome surprises or complications. 87 | 88 | If no implementation has made progress for three months, the document moves to **Expired**. 89 | 90 | If an implementation exposes serious problems that mean the PSC no longer believes the proposal can work, the document moves to **Rejected**. 91 | 92 | If an implementation is delivered and it isn't clear that it's broken as designed, the document moves to **Testing** status. 93 | 94 | During this phase, the proposer or other named implementor is responsible for moving the proposal forward. The PSC will only review proposals in this status when work is delivered or when status updates cease. The PSC will make a regular note of proposals in this status to perl5-porters. The implementors of proposals in this status will post regular updates -- regular enough, at any rate, to avoid becoming Expired. 95 | 96 | ### Testing 97 | 98 | Now there's an accepted proposal and an implementation. At this point, it needs to have enough tests, showing how it will work in real world conditions. The PSC will consult with the author, each other, perl5-porters, and other experts to produce a list of needed test cases. The proposer, implementer, or other volunteers will provide the testing. Once enough testing is done, then the document will be either… 99 | 100 | * marked **Accepted**, with the code merged as a new (probably experimental!) feature of perl 101 | * marked **Rejected**, because the quality or behavior of the feature or its implementation are not acceptable 102 | 103 | If no progress is reported for three months, the document moves to **Expired**. 104 | 105 | During the Testing phase, the PSC and the proposer (or implementor) will be working together and communicating regularly to keep track of what work remains to complete the testing phase. 106 | 107 | ## What needs a PPC? What can just be a PR? 108 | 109 | There's no obvious answer, because there's no clear cut off, and there never will be, even when the process is "out of beta". For now we think we should use PPCs for 110 | 111 | 1. Language changes (feature changes to the parser, tokeniser) 112 | 2. Command line options 113 | 3. Adding/removing warnings (entries in `perldiag.pod`) 114 | 4. Significant changes to when an existing warning triggers 115 | 116 | A case that came up recently was moving the reporting of an error from runtime to compile time (GH #18785). We think that this wouldn't warrant a PPC (just regular code review) because no correct code should be relying on when an error is reported. However, there is still a judgement call here, as it would **not** be correct for constant folding to report errors (such as divide by zero) as this might only happen on some platforms as a side effect of the values of constants, and those expressions were unreachable on those platforms. 117 | -------------------------------------------------------------------------------- /docs/template.md: -------------------------------------------------------------------------------- 1 | # A short and accurate title 2 | 3 | ## Preamble 4 | 5 | Author: A. U. Thor 6 | Sponsor: 7 | ID: ADOPTME-2038 8 | Status: Draft 9 | 10 | The preamble should be RFC-822 like headers. 11 | Author, Sponsor and similar should be valid e-mail addresses, or CPAN IDs. 12 | 13 | ## Abstract 14 | 15 | * 100 to 200 words summarising the entire PPC. 16 | * The most important section is **Motivation**. 17 | 18 | ## Motivation 19 | 20 | * What problem are we trying to solve? 21 | * Why doesn't the existing syntax/functionality cover it? 22 | 23 | ## Rationale 24 | 25 | * Why does the new syntax/functionality solve the problem described above? 26 | * Why choose this solution, and reject others? 27 | 28 | ## Specification 29 | 30 | Perl doesn't have a formal specification. Effectively it's defined by the documentation and the regression tests. Likely the best way to specify a new feature **is** to write the documentation for it. 31 | 32 | You don't need to know the internals to do this - hence the *Author* can contribute directly by working on these, with the *Sponsor* acting as a mentor and guide as needed. 33 | 34 | ## Backwards Compatibility 35 | 36 | If proposing syntax changes, think in terms of "can this be detected by"/"misunderstood by": 37 | 38 | * Static tooling inspecting source code 39 | * The Perl interpreter at compile time 40 | * Only as a runtime error 41 | * Subtle runtime behaviour changes that can't be warned about and break things 42 | 43 | and how does this affect things like 44 | 45 | * [`B::Deparse`](https://metacpan.org/pod/B::Deparse) 46 | * [`Devel::Cover`](https://metacpan.org/pod/Devel::Cover) 47 | * [`Devel::NYTProf`](https://metacpan.org/pod/Devel::NYTProf) 48 | * [`PPI`](https://metacpan.org/pod/PPI) (hence [`Perl::Critic`](https://metacpan.org/pod/Perl::Critic) etc) 49 | 50 | Also, is it possible to emulate this for earlier Perl versions (or at least, a useful and correct subset), even if slow? And if **not**, what sort of API or functionality is missing that if added would make similar future "polyfill"s possible? 51 | 52 | ## Security Implications 53 | 54 | CVEs are not fun. Try to foresee problems. 55 | 56 | ## Examples 57 | 58 | PEPs have this as "How to Teach This". That's a valid goal, but there are different audiences (from newcomers, to experienced Perl programmers unfamiliar with your plan). 59 | 60 | Most of us are not experienced teachers, but many folks reading your PPC are experienced programmers. So probably the best way to demonstrate the benefits of your proposal is to take some existing code in the core or on CPAN (and not your own code) and show how using your new feature can improve it (easier to read, less buggy, etc) 61 | 62 | ## Prototype Implementation 63 | 64 | Is there something that shows the idea is feasible, and lets other people 65 | play with it? Such as 66 | 67 | * A module on CPAN 68 | * A source filter 69 | * Hack the core C code - fails tests, but lets folks play 70 | 71 | ## Future Scope 72 | 73 | Were there any ideas or variations considered that seemed reasonable follow-ons from the initial suggestions, that we don't need to do now, but want to revisit in the future? If there are parts of the design marked as "intended for future expansion", then give an idea here of what direction is planned. 74 | 75 | If that future expansion relies on extending the syntax, then ensure that the first implementation has tests to assert that the extended syntax remains a syntax error. 76 | 77 | ## Rejected Ideas 78 | 79 | Why this solution/this syntax was better than the obvious alternatives. 80 | 81 | We've seen before that there will be **F**requently **A**sked **Q**uestions. 82 | eg *Why not have different behaviour in void context?* 83 | 84 | Likely the answer is in the previous discussion **somewhere**, but most people won't stop to read all the comments. It needs to be easy to find, and updated as it becomes clear which questions are common. 85 | 86 | Hence it **needs** to be in the PPC itself. Without this, the PPC process as a whole won't scale. 87 | 88 | ## Open Issues 89 | 90 | Use this to summarise any points that are still to be resolved. 91 | 92 | ## Copyright 93 | 94 | Copyright (C) 2038, A.U. Thor. 95 | 96 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 97 | -------------------------------------------------------------------------------- /images/flowchart.graffle: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Perl/PPCs/44d06472706c74a1176429e6aa8a5065373a07f0/images/flowchart.graffle -------------------------------------------------------------------------------- /images/flowchart.md: -------------------------------------------------------------------------------- 1 | ```pikchr 2 | # DEFAULTS 3 | 4 | $step = 0.3 5 | linewid = $step 6 | lineht = $step 7 | boxwid = 1.2 8 | boxht = 0.4 9 | 10 | define proposer { box thin fill 0xBFBDFE $1 big rad 10px dashed } 11 | define psc { box thin fill 0xFEFFBF $1 big } 12 | define implementor { box thin fill 0xFFBDFF $1 big rad 10px } 13 | 14 | # MAIN PROCESS 15 | 16 | # proposer 17 | 18 | down 19 | proposer( "cool idea" ) 20 | arrow 21 | proposer( "take it to p5p" ) 22 | right 23 | arrow 24 | proposer( "popular" big "support?" ) 25 | arrow "no" big below 26 | proposer( "give up" ) 27 | down 28 | arrow from 2nd last box.s 29 | proposer( "draft proposal" ) 30 | arrow <-> 31 | 32 | # psc 33 | 34 | Review: psc( "PSC review" ) 35 | 36 | arrow from 2nd last box.w \ 37 | left \ 38 | then down until even with last box \ 39 | then to last box.w <- 40 | 41 | arrow from last box.s 42 | psc( "Exploratory" ) 43 | arrow 44 | text \ 45 | "idea subjected to review" small bold rjust \ 46 | "PSC must approve or reject" small bold rjust \ 47 | at last arrow.w - ($step / 4,0) 48 | 49 | # implementor 50 | 51 | implementor( "Implementing" ) with .n at last arrow.s 52 | arrow 53 | text \ 54 | "implemented by volunteers" small bold rjust \ 55 | "PSC must approve or reject" small bold rjust \ 56 | at last arrow.w - ($step / 4,0) 57 | 58 | arrow from Implementing.s 59 | implementor( "Testing" ) 60 | arrow 61 | text \ 62 | "implementation tested by volunteers" small bold rjust \ 63 | "PSC must approve or reject" small bold rjust \ 64 | at last arrow.w - ($step / 4,0) 65 | 66 | box rad 20px fill 0xBDFFBE "Accepted" big with .n at last arrow.s 67 | 68 | # REJECT 69 | 70 | right 71 | arrow 200% from Implementing.e 72 | RoE: [ 73 | down 74 | box thin "Rejected" big 75 | box thin "or" big italic ht 50% 76 | box thin "Expired" big 77 | ] with .w at last arrow.e 78 | 79 | arrow from Review.s + ( $step, 0 ) to RoE.n - ( $step, 0 ) 80 | arrow from Exploratory.s + ( $step, 0 ) to RoE.w 81 | arrow from Testing.n + ( $step, 0 ) to RoE.w 82 | 83 | # LEGEND 84 | 85 | [ down 86 | boxwid *= 7/8 87 | boxht *= 7/8 88 | WHO: box thin thin "who is responsible?" small fill white fit 89 | move $step/4 90 | proposer( "proposer" ) 91 | move $step/4 92 | psc( "PSC" ) 93 | move $step/4 94 | implementor( "implementor" big "& PSC" ) 95 | box thin width last box.wid + $step height last box.ht * 4.25 \ 96 | with .n at WHO.c behind WHO 97 | ] with .nw at 6th last box.ne + ( linewid * 3.5, $step / 2 ) 98 | ``` 99 | -------------------------------------------------------------------------------- /images/flowchart.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Perl/PPCs/44d06472706c74a1176429e6aa8a5065373a07f0/images/flowchart.png -------------------------------------------------------------------------------- /in/index.tt: -------------------------------------------------------------------------------- 1 |

Proposed Perl Changes

2 | 3 |

Welcome to the Proposed Perl Changes web site.

4 | 5 |

Download this data as JSON.

6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | [% FOR ppc IN ppcs -%] 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | [% END -%] 26 | 27 |
IDTitleStatusAuthor(s)Sponsor
[% ppc.id | html %][% ppc.title | html %][% ppc.status | html %][% ppc.author | html %][% ppc.sponsor | html %]
28 | 29 | 38 | -------------------------------------------------------------------------------- /in/style.css: -------------------------------------------------------------------------------- 1 | main > .container { 2 | padding: 60px 15px 0; 3 | } 4 | -------------------------------------------------------------------------------- /lib/PPC.pm: -------------------------------------------------------------------------------- 1 | use v5.40; 2 | use experimental qw[class signatures]; 3 | 4 | class PPC; 5 | 6 | use Pandoc; 7 | 8 | field $author :param :reader = ''; 9 | field $id :param :reader; 10 | field $slug :param :reader; 11 | field $sponsor :param :reader = ''; 12 | field $status :param :reader; 13 | field $title :param :reader; 14 | 15 | method in_path { 16 | return "$slug.md"; 17 | } 18 | 19 | method out_path { 20 | return "$slug/index.html"; 21 | } 22 | 23 | method as_data { 24 | return { 25 | id => $id, 26 | status => $status, 27 | author => $author, 28 | sponsor => $sponsor, 29 | slug => $slug, 30 | title => $title, 31 | }; 32 | } 33 | 34 | # Very hacky parser 35 | 36 | sub new_from_file($class, $ppc_file) { 37 | 38 | open my $ppc_fh, '<', $ppc_file 39 | or warn "Cannot open PPC [$_]: $!\n" and return; 40 | 41 | my (%ppc, $is_preamble); 42 | 43 | $ppc{slug} = $ppc_file; 44 | $ppc{slug} =~ s|^ppcs/||; 45 | $ppc{slug} =~ s|\.md$||; 46 | 47 | while (<$ppc_fh>) { 48 | !$ppc{title} and m|^#\s+(.*)| and $ppc{title} = md2text($_); 49 | 50 | $is_preamble and /## abstract/i and last; 51 | 52 | $_ = trim($_); 53 | 54 | if ($is_preamble and $_) { 55 | 56 | my ($key, $value) = split(/\s*:\s*/, $_, 2); 57 | 58 | $ppc{lc $key} = $value; 59 | } 60 | 61 | # 'pre.+mble' because Paul likes to use 'Preämble' 62 | /## pre.+mble/i and $is_preamble = 1; 63 | } 64 | 65 | if (exists $ppc{authors}) { 66 | $ppc{author} = delete $ppc{authors} 67 | } 68 | 69 | for (qw[author sponsor]) { 70 | $ppc{$_} //= ''; 71 | $ppc{$_} =~ s|\@.+?>|\@XXXX>|g; 72 | } 73 | 74 | return $class->new(%ppc); 75 | } 76 | 77 | sub md2text($md_string) { 78 | return pandoc->convert( markdown => 'plain', $md_string); 79 | } 80 | -------------------------------------------------------------------------------- /ppcs/ppc0001-n-at-a-time-for.md: -------------------------------------------------------------------------------- 1 | # Multiple-alias syntax for foreach 2 | 3 | ## Preamble 4 | 5 | Author: 6 | Sponsor: Nicholas Clark 7 | ID: 0001 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | Implement the currently illegal syntax `for my ($key, $value) (%hash) { ... }` to act as two-at-a-time iteration over hashes. This approach is not specific to hashes - it generalises to n-at-a-time over any list. 13 | 14 | ## Motivation 15 | 16 | The `each` function is generally discouraged due to implementation details, but is still the most "pretty" and concise way to write such a loop: 17 | 18 | while (my ($key, $value) = each %hash) { ... } 19 | 20 | An alternative to this would be a syntax to alias multiple items per iteration in a `for` loop. 21 | 22 | for my ($key, $value) (%hash) { ... } 23 | 24 | This generalizes as a "bundler" - to alias a number of elements from the list equal to the number of variables specified. Such as: 25 | 26 | for my ($foo, $bar, $baz) (@array) { 27 | # $foo, $bar, and $baz are the next three elements of @array, 28 | # or undef if overflowed 29 | 30 | ## Rationale 31 | 32 | The existing syntax to iterate over the keys and values of a hash: 33 | 34 | while (my ($key, $value) = each %hash) { ... } 35 | 36 | suffers from several problems 37 | 38 | * For correctness it assumes that the internal state of the hash being "clean" - to be robust one should reset the iterator first with `keys %hash;` in void context 39 | * It's hard to teach to beginners - to understand what is going on here one needs to know 40 | - list assignment 41 | - empty lists are false; non-empty lists are true 42 | - that the hash holds an internal iterator 43 | * You can't modify the hash inside the loop without confusing the iterator 44 | 45 | You *can* write it in two lines as 46 | 47 | for my $k (keys %hash) { 48 | my $v = $hash{$h}; 49 | ... 50 | } 51 | 52 | (three if `%hash` is actually a complex expression you don't want to evaluate twice) 53 | 54 | but that's not perceived as an obvious "win" over the one-liner. 55 | 56 | The more general *n-at-a-time* iteration of an array problem doesn't have a simple generic solution. For example, you can **destructively** iterate an **array**: 57 | 58 | while (@list) { 59 | my ($x, $y, $z) = splice @list, 0, 3; 60 | ... 61 | } 62 | 63 | (with that *3* needing to be written out explicitly - it can't derived from the number of lexicals) 64 | 65 | You can iterate over an list non-destructively like this: 66 | 67 | my @temp = qw(Ace Two Three Four Five Six Seven Eight Nine Ten Jack Queen King); 68 | for (my $i = 0; $i < @temp; $i += 3) { 69 | my ($foo, $bar, $baz) = @temp[$i .. $i + 2]; 70 | print "$foo $bar $baz\n"; 71 | } 72 | 73 | but that 74 | 75 | * is verbose 76 | * is hard to get right 77 | * repeats the three-ness in three places 78 | * needs a temporary array (hence a copy) for a list 79 | * also copies all the values into the lexicals 80 | 81 | The proposed syntax solves all of these problems. 82 | 83 | ## Specification 84 | 85 | ``` 86 | diff --git a/pod/perlsyn.pod b/pod/perlsyn.pod 87 | index fe511f052e..490119d00e 100644 88 | --- a/pod/perlsyn.pod 89 | +++ b/pod/perlsyn.pod 90 | @@ -282,6 +282,14 @@ The following compound statements may be used to control flow: 91 | 92 | PHASE BLOCK 93 | 94 | +As of Perl 5.36, you can iterate over multiple values at a time by specifying 95 | +a list of lexicals within parentheses 96 | + 97 | + no warnings "experimental::for_list"; 98 | + LABEL for my (VAR, VAR) (LIST) BLOCK 99 | + LABEL for my (VAR, VAR) (LIST) BLOCK continue BLOCK 100 | + LABEL foreach my (VAR, VAR) (LIST) BLOCK 101 | + LABEL foreach my (VAR, VAR) (LIST) BLOCK continue BLOCK 102 | + 103 | If enabled by the experimental C feature, the following may also be used 104 | 105 | try BLOCK catch (VAR) BLOCK 106 | @@ -549,6 +557,14 @@ followed by C. To use this form, you must enable the C 107 | feature via C. (See L. See also L.) 109 | 110 | +As of Perl 5.36, you can iterate over a list of lexical scalars n-at-a-time. 111 | +If the size of the LIST is not an exact multiple of number of iterator 112 | +variables, then on the last iteration the "excess" iterator variables are 113 | +undefined values, much like if you slice beyond the end of an array. You 114 | +can only iterate over scalars - unlike list assignment, it's not possible to 115 | +use C to signify a value that isn't wanted. This is a limitation of 116 | +the current implementation, and might be changed in the future. 117 | + 118 | Examples: 119 | 120 | for (@ary) { s/foo/bar/ } 121 | @@ -574,6 +590,17 @@ Examples: 122 | # do something which each %hash 123 | } 124 | 125 | + foreach my ($foo, $bar, $baz) (@list) { 126 | + # do something three-at-a-time 127 | + } 128 | + 129 | + foreach my ($key, $value) (%hash) { 130 | + # iterate over the hash 131 | + # The hash is eagerly flattened to a list before the loop starts, 132 | + # but as ever keys are copies, values are aliases. 133 | + # This is the same behaviour as for $var (%hash) {...} 134 | + } 135 | + 136 | Here's how a C programmer might code up a particular algorithm in Perl: 137 | 138 | for (my $i = 0; $i < @ary1; $i++) { 139 | ``` 140 | 141 | ## Backwards Compatibility 142 | 143 | The new syntax is a syntax error on existing Perls. It generates this error message 144 | 145 | `Missing $ on loop variable at /home/nick/test/three-at-a-time.pl line 4.` 146 | 147 | Existing static tooling should be able to recognise it as a syntax error, but without changes wouldn't be able to give any better diagnostics. 148 | 149 | The proposed implementation has tests and patches for `B::Deparse` and `B::Concise`. `Devel::Cover`, `Devel::NYTProf` and `Perl::Critic` handle the new syntax just fine. 150 | 151 | I don't think that we have any way to emulate this syntax for earlier Perls. I don't have any feel for what sort of API we would need to add to the parser to make it possible in the future, and whether any API we could add would be fragile and tightly coupled to the exact current implementation. 152 | 153 | ## Security Implications 154 | 155 | We don't think that this change is likely to cause CVEs or similar problems. 156 | 157 | ## Examples 158 | 159 | The examples in the *Motivation* seem sufficient. 160 | 161 | ## Prototype Implementation 162 | 163 | See https://github.com/nwc10/perl5/commits/smoke-me/nicholas/pp_iter 164 | 165 | ## Future Scope 166 | 167 | ### Permit `undef` in the list of scalars 168 | 169 | for my ($a, undef, $c) (1 .. 9) { ... } 170 | 171 | It's a reasonable generalisation of list assignment, where you can assign to undef to mean "ignore this value". It's also safe syntax to implement for `foreach` (either as part of this proposal, **or** at a later date), and likely will be useful in some specific cases. But **not** having it doesn't hinder the utility of the basic proposal. 172 | 173 | The easiest implementation of *n-at-a-time* `foreach` is if there are exactly *n* scalars, all declared in the `for` loop itself, because this way they occupy adjacent Pad slots. This means that there is only one extra integer to store in the optree, which used both to calculate the *n*-at-a-time **and** the addresses of the target variables. adding `undef` to the mix rules out an obvious simple, clear implementation. 174 | 175 | If we discover that there is a good way to add `undef` without much increased complexity, then we should consider doing this. 176 | 177 | ## Rejected Ideas 178 | 179 | ### Permit @array or %hash in the list of lexicals 180 | 181 | for my ($key, $value, %rest) (%hash) { ... } 182 | for my ($first, $second, @rest) (@array) {... } 183 | 184 | Syntactically these all "work", and don't violate the assumption that all lexicals are in adjacent Pad slots. But it would add complexity to the runtime. Generalising *1 scalar at a time* to *n at a time* is mostly just adding a C `for` loop around some existing (known working) code. 185 | 186 | Implementing these would mean adding code for what is basically a funky way of writing 187 | 188 | { my ($key, $value, %rest) = %hash; ... } 189 | { my ($first, $second, @rest) = @array; ... } 190 | 191 | ### Permit `our` as well as `my` 192 | 193 | Using `our` instead of `my` works in the grammar. ie we can parse this: 194 | 195 | for our ($foo, $bar, $baz) (@array) { 196 | 197 | However, the implementation would be far more complex. Each package variable 198 | would need a pair of GV and RV2GV ops to set it up for aliasing in a lexical, 199 | meaning that we'd need to create a list to feed into ENTERITER, and have it 200 | process that list to set up Pads. This complexity isn't worth it. 201 | 202 | ### Alternative parsable syntaxes suggested by Father Chrysostomos 203 | 204 | When this syntax was suggested in 2017, Father Chrysostomos observed that in addition to the first syntax, the next two are also currently errors and could be parsed: 205 | 206 | for my ($key, $value) (%hash) { ... } 207 | for my $key, my $value (%hash) { ... } 208 | for $foo, $bar (%hash) { ... } 209 | 210 | He notes that it's only this illegal syntax that could not be parsed: 211 | 212 | for ($foo, $bar) (%hash) { ... } # syntax error 213 | 214 | https://www.nntp.perl.org/group/perl.perl5.porters/2017/04/msg243856.html 215 | 216 | Strictly we also can't parse this: 217 | 218 | for (my $key, my $value) (%hash) { ... } 219 | 220 | meaning that we can't offer choice of syntax in `for` analogous to: 221 | 222 | (my $foo, my $bar, my $baz) = @ARGV 223 | my ($foo, $bar, $baz) = @ARGV 224 | 225 | which parse to the same optree. 226 | 227 | However, these two lines are **not the same**: 228 | 229 | my $key, my $value = @pair; 230 | my ($key, $value) = @pair; 231 | 232 | hence we should not offer the alternative syntax: 233 | 234 | for my $key, my $value (%hash) { ... } 235 | 236 | For implementation reasons we only want to handle lexicals. This rules out: 237 | 238 | for $foo, $bar (%hash) { ... } 239 | 240 | So these alternative possible syntaxes should not be offered. 241 | 242 | ### Should this be behind a feature guard? 243 | 244 | As the new syntax was previously an error, and previously in similar situations we have not used a feature guard, we don't think that we should this time. 245 | 246 | ## Open Issues 247 | 248 | ## Copyright 249 | 250 | Copyright (C) 2021, Nicholas Clark 251 | 252 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 253 | -------------------------------------------------------------------------------- /ppcs/ppc0003-np-warning.md: -------------------------------------------------------------------------------- 1 | # Issue a warning "-np better written as -p" 2 | 3 | ## Preamble 4 | 5 | Author: 6 | Sponsor: Nicholas Clark 7 | ID: 0003 8 | Status: Rejected 9 | 10 | ## Abstract 11 | 12 | `perlrun` says *A **-p** overrides a **-n** switch.* 13 | 14 | We warn for other overrides, such as variables declared twice. To be consistent we should warn for this one, if `-w` is on the command line. 15 | 16 | ## Motivation 17 | 18 | We explicitly document that `-p` overrides `-n`. Calling the `perl` binary with both is not correct - the `-n` will be ignored. We could help users better by reporting their mistake to them, if they have opted into warnings. 19 | 20 | ## Rationale 21 | 22 | For code, where what is written cannot make sense, we issue warnings. This is a similar case, just with command line flags 23 | 24 | * Issuing a warning would make a programmer aware of the problem 25 | * Issuing a warning would be consistent with our other use of warnings 26 | 27 | ## Specification 28 | 29 | Invoking `perl` with all three of `-p`, `-w` and `-n` in any order or grouping should issue the warning 30 | 31 | -np better written as -p 32 | 33 | ## Backwards Compatibility 34 | 35 | This is **hard** to assess. 36 | 37 | We can search CPAN for representative use of Perl **code**. With the demise of Google codesearch, there isn't a good way to search for command-line use cases of `perl`. Is it viable to search Debian's source archives? Or the FreeBSD ports tree? 38 | 39 | Issuing a warning **might** break existing users' code, and they would be grumpy, because it was working, it would still work without a trapped warning, and we have no intention of changing the behaviour 40 | 41 | It might "break" existing code, where users view "you're making new noise" as breakage, but (of course) everything still works. 42 | 43 | It might not make much difference - do we have any feel for how many scripts invoking `perl` as a command-line *better sed*/*better awk* actually use `-w` **at all**? 44 | 45 | ## Security Implications 46 | 47 | It's unclear whether there is any (direct) security implication. 48 | 49 | ## Examples 50 | 51 | I believe that it's exhaustively covered in the *Specification*. 52 | 53 | ## Future Scope 54 | 55 | We don't intend to make this warning "fatal". 56 | 57 | ## Rejected Ideas 58 | 59 | The PSC thanks the author for making a useful suggestion, but has decided that this change is not worth making. 60 | 61 | The discussion was useful. We note 62 | 63 | 1) https://codesearch.debian.net/ lets one search for typical command-line invocations of `perl`. This is useful when considering changes to options or option parsing. 64 | 2) Command line options are processed as a result of all of 65 | 1) Actual command line options 66 | 2) Options found on a `#!` line 67 | 3) Options in `PERL5OPT` 68 | 69 | Hence "combinations" of options might have happened as a result of the internal unification of these, not because the programmer wrote something directly. 70 | 71 | Using `-p` and `-n` is not causing one to be ignored. "-p" is "-n and also more". The gain to not writing both is saving a single (or perhaps 2-3) keystrokes. If the user has written both, their program will work just as well as one with only one. Removing one would require **more** work. Also, we'd need a warning message sufficiently clear that the user knew, immediately, to go remove the "n", but also that no change was really **required**. The benefit is very close to nil. 72 | 73 | This is **not** the same as the `@F[1]` warning. A simple test case doesn't suggest this: 74 | 75 | $ perl -wE 'our @A = (1,2,3); say @A[0]' 76 | Scalar value @A[0] better written as $A[0] at -e line 1. 77 | 1 78 | 79 | But there is a massively important one that's demonstrated by a different code snippet: 80 | 81 | $ perl -wE 'our @A; @A[0] = foo(); sub foo { warn wantarray ? "1\n" : "0\n" }' 82 | 1 83 | $ perl -wE 'our @A; $A[0] = foo(); sub foo { warn wantarray ? "1\n" : "0\n" }' 84 | 0 85 | 86 | Using `@A[0]` creates a list context. Using `$A[0]` creates a scalar context. It's a setup for potentially deep confusion based on sigil variance, worth the warning. 87 | 88 | ## Open Issues 89 | 90 | ## Copyright 91 | 92 | Copyright (C) 2021, Nicholas Clark 93 | 94 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 95 | -------------------------------------------------------------------------------- /ppcs/ppc0004-defer-block.md: -------------------------------------------------------------------------------- 1 | # Deferred block syntax 2 | 3 | ## Preämble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0004 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | Add a new control-flow syntax written as `defer { BLOCK }` which enqueues a block of code for later execution, when control leaves its surrounding scope for whatever reason. 13 | 14 | ## Motivation 15 | 16 | Sometimes a piece of code performs some sort of "setup" operation, that requires a corresponding "teardown" to happen at the end of its task. Control flow such as exception throwing, or loop controls, means that it cannot always be written in a simple style such as 17 | 18 | ```perl 19 | { 20 | setup(); 21 | operation(); 22 | teardown(); 23 | } 24 | ``` 25 | 26 | as in this case, if the `operation` function throws an exception the teardown does not take place. Traditional solutions to this often rely on allocating a lexical variable and storing an instance of some object type with a `DESTROY` method, so this runs the required code as a result of object destruction caused by variable cleanup. There are no in-core modules to automate this process, but the CPAN module `Scope::Guard` is among the more popular choices. 27 | 28 | It would be nice to offer a native core syntax for this common behaviour. A simple `defer { BLOCK }` syntax removes from the user any requirement to think about storing the guard object in a lexical variable, or worry about making sure it really does get released at the right time. 29 | 30 | This syntax has already been implemented as a CPAN module, [`Syntax::Keyword::Defer`](https://metacpan.org/pod/Syntax::Keyword::Defer). This PPC formalizes an attempt implement the same in core. 31 | 32 | ## Rationale 33 | 34 | ((TODO - I admit I'm not very clear on how to split my wording between the Motivation section, and this one)) 35 | 36 | The name "defer" comes from a collection of other languages. Near-identical syntax is provided by Swift, Zig, Jai, Nim and Odin. Go does define a `defer` keyword that operates on a single statement, though its version defers until the end of the containing function, not just a single lexical block. I did consider this difference, but ended up deciding that a purely-lexical scoped nature is cleaner and more "Perlish", overriding the concerns that it differs from Go. 37 | 38 | ## Specification 39 | 40 | A new lexically-scoped feature bit, requested as 41 | 42 | ```perl 43 | use feature 'defer'; 44 | ``` 45 | 46 | enables new syntax, spelled as 47 | 48 | ```perl 49 | defer { BLOCK } 50 | ``` 51 | 52 | This syntax stands alone as a full statement; much as e.g. a `while` loop does. The deferred block may contain one or multiple statements. 53 | 54 | When the `defer` statement is encountered during regular code execution, nothing immediately happens. The contents of the block are stored by the perl interpreter, enqueued to be invoked at some later time, when control exits the block this `defer` statement is contained within. 55 | 56 | If multiple `defer` statements appear within the same block, the are eventually executed in LIFO order; that is, the most recently encountered is the first one to run: 57 | 58 | ```perl 59 | { 60 | setup1(); 61 | defer { say "This runs second"; teardown1(); } 62 | 63 | setup2(); 64 | defer { say "This runs first"; teardown2(); } 65 | } 66 | ``` 67 | 68 | `defer` statements are only "activated" if the flow of control actually encounters the line they appear on (as compared to `END` phaser blocks which are activated the moment they have been parsed by the compiler). If the `defer` statement is never reached, then its deferred code will not be invoked: 69 | 70 | ```perl 71 | while(1) { 72 | defer { say "This will happen"; } 73 | last; 74 | defer { say "This will *NOT* happen"; } 75 | } 76 | ``` 77 | 78 | ((TODO: It is currently not explicitly documented, but naturally falls out of the current implementation of both the CPAN and the in-core versions, that the same LIFO stack that implements `defer` also implements other things such as `local` modifications; for example: 79 | 80 | ```perl 81 | our $var = 1; 82 | { 83 | defer { say "This prints 1: $var" } 84 | 85 | local $var = 2; 86 | defer { say "This prints 2: $var" } 87 | } 88 | ``` 89 | 90 | ((I have no strong thoughts yet on whether to specify and document - and thus guarantee - this coïncidence, or leave it unsaid.)) 91 | 92 | Non-local control flow (`next/last/redo`, `goto`, `return`) is not permitted to cross the boundary of the `defer` block 93 | 94 | ```perl 95 | foreach my $item (@things) { 96 | defer { last; } ## This is NOT permitted 97 | } 98 | ``` 99 | 100 | Attempts to do this will throw an exception, likely worded something about 101 | 102 | ``` 103 | Attempting to 'last' out of a 'defer' block is not permitted at FILE line LINE. 104 | ``` 105 | 106 | Of course, nonlocal control flow is permitted entirely *within* the `defer` block; we only prohibit control flow that attempts to leave the block: 107 | 108 | ```perl 109 | { 110 | defer { 111 | foreach my $i ( 1 .. 10 ) { 112 | last if $i == 5; ## This is permitted 113 | } 114 | } 115 | } 116 | ``` 117 | 118 | The one exception to this (pardon the pun) is that exceptions thrown from within the `defer` block propagate out to the caller. 119 | 120 | For example, in 121 | 122 | ```perl 123 | sub f { 124 | defer { die "This is thrown\n"; } 125 | } 126 | 127 | f(); 128 | ``` 129 | 130 | the caller will see the exception being thrown in the same way as if it appeared in a regular (non-`defer`red) block. 131 | 132 | It is currently unspecified what happens if a `defer` block throws an exception while unwinding the stack because of an exception. 133 | 134 | ```perl 135 | { 136 | defer { die "Exception A\n"; } 137 | die "Exception B\n"; 138 | } 139 | ``` 140 | 141 | In this case, the caller will definitely see *an* exception thrown from the code, but there is currently no guarantee whether that will be A, B, or some third kind of thing that is a combination of the two. This is intentionally left as scope for future expansion; at such time perl ever gains true object-like core exceptions, then it can be represented by some kind of "double exception" condition. 142 | 143 | ## Backwards Compatibility 144 | 145 | The new syntax `defer { BLOCK }` is guarded by a lexical feature guard. A static analyser that is not aware of that feature guard would get confused into thinking this is indirect call syntax; though this is no worse than basically any other feature-guarded keyword that controls a block (e.g. `try/catch`). 146 | 147 | For easy compatibility on older Perl versions, the CPAN implementation already mentioned above can be used. If this PPC succeeds at adding it as core syntax, a `Feature::Compat::Defer` module can be created for it in the same style as [`Feature::Compat::Try`](https://metacpan.org/pod/Feature::Compat::Try). 148 | 149 | ## Security Implications 150 | 151 | None foreseen. 152 | 153 | ## Examples 154 | 155 | A couple of small code examples are quoted above. Further, from the docs of `Syntax::Keyword::Defer`: 156 | 157 | ```perl 158 | use feature 'defer'; 159 | 160 | { 161 | my $dbh = DBI->connect( ... ) or die "Cannot connect"; 162 | defer { $dbh->disconnect; } 163 | 164 | my $sth = $dbh->prepare( ... ) or die "Cannot prepare"; 165 | defer { $sth->finish; } 166 | 167 | ... 168 | } 169 | ``` 170 | 171 | ## Prototype Implementation 172 | 173 | CPAN module `Syntax::Keyword::Defer` as already mentioned. 174 | 175 | In addition, I have a mostly-complete branch of bleadperl (somewhat behind since I haven't updated it for the 5.34 release yet) at 176 | 177 | https://github.com/leonerd/perl5/tree/defer 178 | 179 | I'm not happy with the way I implemented it yet (don't look at how I abused an SVOP to store the deferred optree) - it needs much tidying up and fixing for the various things I learned while writing the CPAN module version. 180 | 181 | ## Future Scope 182 | 183 | If this PPC becomes implemented, it naturally follows to enquire whether the same mechanism that powers it could be used to add a `finally` clause to the `try/catch` syntax added in Perl 5.34. This remains an open question: while it doesn't any new ability, is the added expressive power and familiarity some users will have enough to justify there now being two ways to write the same thing? 184 | 185 | Additionally, once `defer` exists and if core exceptions ever gain some sort of metadata or object-like representation beyond simple strings, then the double-exception case mentioned above can be addressed. 186 | 187 | ## Rejected Ideas 188 | 189 | On the subject of naming, this was originally called `LEAVE {}`, which was rejected because its semantics don't match the Raku language feature of the same name. It was then renamed to `FINALLY {}` where it was implemented on CPAN. This has been rejected too on grounds that it's too similar to the proposed `try/catch/finally`, and it shouldn't be a SHOUTY PHASER BLOCK. All the SHOUTY PHASER BLOCKS are declarations, activated by their mere presence at compiletime, whereas `defer {}` is a statement which only takes effect if dynamic execution actually encounters it. The p5p mailing list and the CPAN module's RT queue both contain various other rejected naming ideas, such as UNWIND, UNSTACK, CLEANUP, to name three. 190 | 191 | Another rejected idea is that of conditional enqueue: 192 | 193 | ```perl 194 | defer if (EXPR) { BLOCK } 195 | ``` 196 | 197 | as this adds quite a bit of complication to the grammar, for little benefit. As currently the grammar requires a brace-delimited block to immediately follow the `defer` keyword, it is possible that other ideas - such as this one - could be considered at a later date however. 198 | 199 | ## Open Issues 200 | 201 | Design-wise I don't feel there are any remaining unresolved questions. 202 | 203 | Implementation-wise the code still requires some work to finish it off; it is not yet in a merge-ready state. 204 | 205 | ## Copyright 206 | 207 | Copyright (C) 2021, Paul Evans. 208 | 209 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 210 | -------------------------------------------------------------------------------- /ppcs/ppc0005-everything-slices.md: -------------------------------------------------------------------------------- 1 | # Everything Slices 2 | 3 | ## Preamble 4 | 5 | Author: Ricardo Signes 6 | Sponsor: Ricardo Signes 7 | ID: 0005 8 | Status: Rejected 9 | 10 | ## Abstract 11 | 12 | Perl's slice syntax lets the programmer get at a subset of values or key/value 13 | pairs in a hash or array. This PPC proposes a syntax get a slice that contains 14 | the complete set of values or key/values in the hash or array. 15 | 16 | ## Motivation 17 | 18 | It's always possible to get a slice of everything in a container, but it 19 | requires writing the expression that expresses "all the keys". This duplicates 20 | the variable name, which is an opportunity for error in both reading and 21 | writing. 22 | 23 | The nearest motivation for this change is the potential addition of n-at-a-time 24 | iteration in foreach. This syntax will work by default: 25 | 26 | for my ($k, $v) (%hash) { ... } 27 | 28 | To iterate over an array's indexes and values, one would have to write: 29 | 30 | for my ($i, $v) (%array[ keys @array ]) { ... } 31 | 32 | Note, too, the sigil variance. 33 | 34 | ## Rationale 35 | 36 | `%hash` in list context evaluates to a list of pairs, while `@array` in list 37 | context evalutes to the values. There's a means to get the values of a hash 38 | with `values`, but no existing built-in for (say) `kv @array`. Rather than 39 | propose adding a `kv`, which would only be useful on arrays, this PPC proposes 40 | adding a type of slice that includes all keys and values. 41 | 42 | %hash{*}; # equivalent to %hash{ keys %hash } 43 | %array[*]; # equivalent to %array[ 0 .. $#array ] 44 | 45 | @hash{*}; # equivalent to @hash{ keys %hash } 46 | @array[*]; # equivalent to @array[ 0 .. $#array ] 47 | 48 | Note that `@array[*]` is *not* equivalent to `@array`, because it is a *slice* 49 | rather than a list of values. This is especially notable when taking a 50 | reference or performing assignment. 51 | 52 | \@array; # yields a reference to the array 53 | \@array[*]; # yields a list of references, one to each entry in the array 54 | 55 | @array = (1 .. 100); # the list is now 100 elements long 56 | @array[*] = (1 .. 100); # the list length is unchanged 57 | 58 | The availability of `%array[*]` is the primary reason to add this feature, but 59 | I have proposed this slice mechanism rather than a `kv` built-in because it 60 | provides straightforward syntax to more easily produce structures already 61 | familiar to at least intermediate Perl programmers. 62 | 63 | The use of `*` is cribbed from Raku, where it is used a a multipurpose 64 | placeholder. While I believe this use is a plausible starting point for future 65 | expansion of the asterisk semantics, we could stop here without causing undue 66 | confusion. 67 | 68 | ## Specification 69 | 70 | The token `*` is permitted to stand alone (with optional whitespace on either 71 | or both sides) inside the `{...}` or `[...]` using in slicing a hash or array. 72 | A slice using `*` in place of subscript keys will act as if all keys were 73 | provided. For arrays, results will be provided in the numeric order of the 74 | keys. This may be optimized, but need not be. 75 | 76 | Assignment to a key/value air everything-slice is forbidden, as it is for any 77 | other key/value slice. Assignment to a value-only everything-slice is 78 | permitted as usual. 79 | 80 | The asterisk may not be used as part of a larger expression inside the 81 | subscript. For example, this is not legal: 82 | 83 | %array[ grep {; $_ > 5 } * ] 84 | 85 | The behavior of an everything-slice on a tied array should be identical to the 86 | behavior of: 87 | 88 | @array[ keys @array ]; 89 | 90 | The behavior of an everything-slice on a tied hash should be identical to the 91 | behavior of: 92 | 93 | @hash{ keys %hash }; 94 | 95 | ## Backwards Compatibility 96 | 97 | The use of `*` as a standalone subscript is already a syntax error, and could 98 | be introduced without requiring any other changes or deprecations. 99 | 100 | I believe that updating static analyzers will not be extremely complex, but I'm 101 | not particularly expert in any of them. 102 | 103 | The deparser and coverage tools may need updating, but presumably no more than 104 | many other small changes. (This does not introduce any new runtime branching 105 | behavior.) 106 | 107 | I can't speak to the effect on Devel::NYTProf at all. 108 | 109 | This can't be easily implemented in older versions of perl. 110 | 111 | ## Security Implications 112 | 113 | [ none yet foreseen ] 114 | 115 | ## Examples 116 | 117 | [ can produce ] 118 | 119 | ## Prototype Implementation 120 | 121 | Is there something that shows the idea is feasible, and lets other people 122 | play with it? Such as 123 | 124 | * A module on CPAN 125 | * A source filter 126 | * Hack the core C code - fails tests, but lets folks play 127 | 128 | ## Future Scope 129 | 130 | If we adopt `*` as a "everything" placeholder, we may want to use it in more 131 | places, possibly to be investigated by skimming off the top of Raku when 132 | applicable. 133 | 134 | ## Rejected Ideas 135 | 136 | ### Just Add kv 137 | 138 | The simplest alternative here is to simply add a new built-in, `kv`, which 139 | operates on an array and evalutes to a list of index/value pairs, or operates 140 | on a hash, returning the key/value pairs. 141 | 142 | This is much simpler semantically and, presumably, in implementation. On the 143 | other hand, it has somewhat fewer potential applications. 144 | 145 | ### The Empty Subscript Option 146 | 147 | One alternative to the asterisk that was proposed in the past was using an 148 | empty subscript. For example: 149 | 150 | # Instead of: 151 | @array[*] 152 | 153 | # One could write: 154 | @array[] 155 | 156 | I believe this looks much more like a mistake. Moreover, note that while 157 | `@array[]` is currently illegal, `@array[()]` is legal, and evaluates to an 158 | empty list. I think this would lead to confusion. (Also, while I do not want 159 | to act as if generated code is a primary target, I think this does needlessly 160 | complicated code generation.) 161 | 162 | ### The Endless Range Option 163 | 164 | Another alternative syntax is an extended range operator. These are already 165 | legal: 166 | 167 | @array[ 2 .. 3 ] 168 | 169 | @array[ 0 .. 99 ] 170 | 171 | @array[ 0 .. $#array ] 172 | 173 | The proposal is that, at least in the context of an array subscript, the ends 174 | could be left off. That is: 175 | 176 | @array[ .. 99 ] # equivalent to 0..99 177 | 178 | @array[ 0.. ] # equivalent to 0 .. $#array 179 | @array[ .. ] # equivalent to 0 .. $#array 180 | 181 | There may be further ambiguity here, but I have mostly ignored this option 182 | because its applicability to hashes seems weird to me. Hash keys are not 183 | ordered, and can't be ranged. Only the bare `..` would be useful: 184 | 185 | %hash{..} # equivalent to %hash{ keys %hash }, currently a syntax error 186 | 187 | That said, the implicit-ended range operator may have more direct uses than the 188 | asterisk. 189 | 190 | ## Open Issues 191 | 192 | Use this to summarise any points that are still to be resolved. 193 | 194 | ## Copyright 195 | 196 | Copyright (C) 2021, Ricardo Signes. 197 | 198 | This document and code and documentation within it may be used, redistributed 199 | and/or modified under the same terms as Perl itself. 200 | -------------------------------------------------------------------------------- /ppcs/ppc0006-load-module.md: -------------------------------------------------------------------------------- 1 | # Module Loading with "load\_module" 2 | 3 | ## Preamble 4 | 5 | Author: Ricardo Signes 6 | Sponsor: Ricardo Signes 7 | ID: 0006 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | This PPC proposes a new built-in, "load\_module", to load modules by module 13 | name (rather than filename) at runtime. 14 | 15 | ## Motivation 16 | 17 | Perl's `require` built-in will interpret a string argument as a filename and 18 | attempt to load this filename in @INC. Often, a Perl programmer would like to 19 | load a module by module name at runtime. That is, they want to pass `require` 20 | a variable containing `Foo::Bar` rather than `Foo/Bar.pm`. To solve this, a 21 | number of libraries have been published to the CPAN with various quirks, extra 22 | behaviors, and bugs. This has led to more than one run of "everyone should 23 | switch from using loader A to loader B." 24 | 25 | ## Rationale 26 | 27 | This PPC proposes exactly one behavior: load a named module by module name or 28 | die trying. Putting this in the core should allow using exactly the same 29 | loading logic as the core uses. Avoiding any further behaviors or add-ons 30 | eliminates argument about what or how. 31 | 32 | ## Specification 33 | 34 | ```pod 35 | =item load_module EXPR 36 | 37 | This loads a named module from the inclusion paths (C<@INC>). EXPR must be 38 | a string that provides a module name. It cannot be omitted, and providing 39 | an invalid module name will result in an exception. 40 | 41 | The effect of C-ing a module is the same as C-ing, down 42 | to the same error conditions when the module does not exist, does not compile, 43 | or does not evalute to a true value. 44 | 45 | C can't be used to require a particular version of Perl, nor can 46 | it be given a bareword module name as an argument. 47 | 48 | C is only available when requested. 49 | ``` 50 | 51 | Note: The "how" of "when requested" is to be determined. If implemented 52 | today, it would be provided behind "use feature". [Current p5p 53 | discussion](https://github.com/Perl/PPCs/blob/master/ppcs/ppc0009.md) suggests 54 | that in the future it would be imported with std.pm or builtin.pm. 55 | 56 | ## Backwards Compatibility 57 | 58 | No backward compatibility concerns are identified here. The deparser and other 59 | tools will need updating. It should be fairly possible to provide a polyfill 60 | for this behavior, with some minor rough edges, by falling back to pure Perl 61 | module name validation and eval or filename-based require. 62 | 63 | ## Security Implications 64 | 65 | The major security problem to be considered is arbitrary code loading. If the 66 | module name validation is robust, then we should see the same guarantees as 67 | with `require`. 68 | 69 | ## Examples 70 | 71 | For examples, consider Module::Load, Class::Load, Module::Runtime, and others. 72 | A simple use case might be: 73 | 74 | ```perl 75 | my $config = read_config; 76 | for my $module (keys $config->{prereq}->%*) { 77 | my ($version) = $config->{prereq}{$module}; 78 | load_module $module; 79 | $module->VERSION($version) if defined $version; 80 | } 81 | ``` 82 | 83 | ## Prototype Implementation 84 | 85 | Numerous prototype implementations exist, in effect. This PPC is the request 86 | to build the real deal. 87 | 88 | ## Future Scope 89 | 90 | One feature seen in several module loaders is "try to load, returning false if 91 | it fails" or variations on that theme. These, among other things, allow 92 | differentiation between failure to find the file, failure to compile, and other 93 | errors. 94 | 95 | These problems already exist with `use` and `require`. Addressing them through 96 | more clearly-defined exceptions has been proposed in the past, and this PPC has 97 | not attempted to fix them just for `load_module`. Instead, a future fix should 98 | address all at once. 99 | 100 | ## Rejected Ideas 101 | 102 | A second argument to load, to specify a minimum version, has not been included. 103 | This is already covered by calling `$module->VERSION($version)` and would be a 104 | further distinction from `require`. 105 | 106 | ## Open Issues 107 | 108 | * how is the feature made available? 109 | 110 | ## Copyright 111 | 112 | Copyright (C) 2021, Ricardo Signes 113 | 114 | This document and code and documentation within it may be used, redistributed 115 | and/or modified under the same terms as Perl itself. 116 | -------------------------------------------------------------------------------- /ppcs/ppc0007-source-encoding.md: -------------------------------------------------------------------------------- 1 | # source encoding pragma 2 | 3 | ## Preamble 4 | 5 | Author: Ricardo Signes 6 | Sponsor: Ricardo Signes 7 | ID: 0007 8 | Status: Draft 9 | 10 | ## Abstract 11 | 12 | This PPC proposes a new pragma, `source::encoding`, to indicate the encoding of 13 | the source document. 14 | 15 | ## Motivation 16 | 17 | At present, unless in a scope in which `use utf8` has been enabled, bytes read 18 | from source correspond directly to the codepoints as which they are 19 | interpreted. This leads to some surprising behaviors. A Latin-1 encoded 20 | source file will have its literal strings match Unicode semantics when matching 21 | regular expressions. Meanwhile, a UTF-8 encoded source file's strings may not 22 | appear to do so, but will behave correctly when printed to a UTF-8 terminal. 23 | 24 | All these behaviors can be explained, but can still surprise both beginner and 25 | expert. To eliminate surprise at runtime, this proposal intends to give the 26 | programmer a means to declare that non-ASCII bytes are a compile-time error. 27 | It also proposes to make that declaration implicit in "use v5.38" and later 28 | version declarations. 29 | 30 | ## Rationale 31 | 32 | The biggest goal here is to make "use v5.38" sufficient to avoid runtime 33 | confusion falling out of non-ASCII source. Given the complexity of "just make 34 | it all Unicode and UTF-8", the goal is to alert the programmer that they've 35 | used non-ASCII in their source without declaring that they've thought about it. 36 | 37 | The behaviors implied by "use VERSION" are generally made individually 38 | controllable, so a separate control must be provided. Rather than provide a 39 | "use ascii" that parallels "use utf8", a single "use source::encoding" is 40 | provided so that a common name can be used for both declaring ASCII-only and 41 | UTF-8 encoding. 42 | 43 | ## Specification 44 | 45 | A new pragma will be created, source::encoding, which can be given one of two 46 | arguments: `utf8` or `ascii`. 47 | 48 | `use source::encoding "utf8"` will have the same effect as `use utf8`. 49 | 50 | `use source::encoding "ascii"` will indicate that a compile-time error should 51 | be raised when reading a non-ASCII byte in the source. 52 | 53 | `no source::encoding` will return to the default behavior of reading bytes into 54 | codepoints. 55 | 56 | `use v5.38` (and later) will implicitly set the source encoding to ASCII. 57 | Using the feature bundle will have no effect on source encoding. 58 | 59 | ## Backwards Compatibility 60 | 61 | Static analysis that currently attempts to detect `use utf8` will need to 62 | be updated to also detect `use source::encoding ARG`. This creates a 63 | significant complication, because ARG can be a variable. On the other hand, 64 | the problem is already quite difficult, because any library loaded at compile 65 | time could affect the encoding of the scope currently being compiled, by dint 66 | of how `$^H` works. 67 | 68 | The source::encoding library can be backported to earlier perls, but only for 69 | utf8, not ascii, unless a source filter is used -- which may actually be a 70 | reasonable use case for source filtering. 71 | 72 | ## Security Implications 73 | 74 | None foreseen. 75 | 76 | ## Examples 77 | 78 | Producing examples where non-ASCII source leads to confusion is like shooting 79 | fish in a barrel. 80 | 81 | use strict; 82 | use feature 'say'; 83 | 84 | my $string = "Queensrÿche"; 85 | 86 | say length $string; 87 | say "contains non-words" if $string =~ /\W/; 88 | say $string; 89 | 90 | Many different problems may arise if the source is encoded as Latin-1 versus 91 | UTF-8, whether `use utf8` is inserted, whether `use feature "unicode_strings"` 92 | is enabled, and so on. 93 | 94 | By adding `use source::encoding "ascii"`, all (or nearly all) of those problems 95 | are replaced by the simple question of "How shall we represent the 8th position 96 | of that string in the source code?" 97 | 98 | ## Prototype Implementation 99 | 100 | It would be possible to implement this with a source filter, but the author has 101 | not attempted to do so. 102 | 103 | ## Future Scope 104 | 105 | In the future, if the semantics of strings and filehandles are expanded to 106 | better cover encoding issues, it may become practical to change the `use vX` 107 | behavior to prefer UTF-8 to ASCII. 108 | 109 | ## Rejected Ideas 110 | 111 | The first proposal to eliminate non-ASCII source footguns was to have use v5.38 112 | enable the utf8 pragma. The objection raised was that this would lead to new 113 | kinds of confusion related to decoded (text) strings being printed to 114 | filehandles with no encoding layer. Although this is explainable behavior, the 115 | current behavior may be less confusing in its practical effect, in some 116 | circumstances. Instead, "demand ASCII" has been proposed because it keeps 117 | semantics within the common space of ASCII, Latin-1, and UTF-8. 118 | 119 | The `encoding` pragma once performed a very similar task to the new proposed 120 | pragma, but for arbitrary encodings. It has been unsupported for several 121 | years. Reclaiming this name for this function is tempting, but seems likely to 122 | cause confusion. 123 | 124 | ## Open Issues 125 | 126 | None? 127 | 128 | ## Copyright 129 | 130 | Copyright (C) 2021, Ricardo Signes 131 | 132 | This document and code and documentation within it may be used, redistributed 133 | and/or modified under the same terms as Perl itself. 134 | -------------------------------------------------------------------------------- /ppcs/ppc0008-sv-boolean-type.md: -------------------------------------------------------------------------------- 1 | # Stable SV Boolean Type 2 | 3 | ## Preamble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0008 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | Add to regular (defined and non-referential) scalars the ability to track whether they represent a boolean value. Values created from boolean expressions (such as `$x == $y`) will reliably remember this fact, in a way that can be introspected even if the value is stored into a variable and retrieved later. 13 | 14 | ## Motivation 15 | 16 | Language interoperability concerns sometimes lead to situations where it is necessary to represent typed data - such as strings or numbers - in a way that can be reliably distinguished. Elsewhere in Perl there are attempts to add this distinction. This PPC attempts to address how to handle values of a boolean type; values that represent a simple true-or-false nature. 17 | 18 | For example, JSON and MsgPack are commonly-encountered serialisation formats that Perl can generate and parse, though in both cases while the formats themselves can represent boolean truth values distinctly from numbers or strings, perl cannot preserve that distinction. Compare this to some other languages popular at the moment: 19 | 20 | ``` 21 | $ python3 -c 'import json; print(json.dumps([1, "1", 1 == 1]))' 22 | [1, "1", true] 23 | 24 | $ nodejs -e 'console.log(JSON.stringify([1, "1", 1 == 1]))' 25 | [1,"1",true] 26 | 27 | $ perl -MJSON::MaybeUTF8=encode_json_utf8 -E 'say encode_json_utf8([1, "1", 1 == 1])' 28 | [1,"1",1] 29 | ``` 30 | 31 | There is a tendancy for modules to make up this shortfall with workarounds like the `JSON::PP::Boolean` type: 32 | 33 | ``` 34 | $ perl -MJSON::MaybeUTF8=decode_json_utf8 -MData::Dump=pp 35 | -E 'say pp(decode_json_utf8($ARGV[0]))' '[1,true]' 36 | 37 | [1, bless(do{\(my $o = 1)}, "JSON::PP::Boolean")] 38 | ``` 39 | 40 | Obviously such a solution is specific to JSON encoding and does not apply to, for example, message gateway between JSON and MsgPack, which would require some translation inbetween. A true in-core solution to this problem would have many benefits to interoperability of data handling between these various modules. 41 | 42 | ((TODO: Add some comments about purely in-core handling as well that don't rely on serialisation)) 43 | 44 | ## Rationale 45 | 46 | ## Specification 47 | 48 | There are two parts to this specification; the lower-level in-core details of the C code implementation that are visible to XS code; and the higher-level Perl-visible details that are made visible to Perl programs. 49 | 50 | ### C-level Internals 51 | 52 | At the C level, SVs will require a macro to test whether they contain a boolean. 53 | 54 | ``` 55 | SvIsBOOL(sv) 56 | ``` 57 | 58 | The core immortals `PL_sv_yes` and `PL_sv_no` will always respond true to `SvIsBOOL()`, as will any SV initialised by copying from them by using `sv_setsv()` and friends. The upshot here is that the result of boolean-returning ops will be `SvIsBOOL()` and this flag will remain with any copies of that value that get made - either over the arguments stack or stored in lexicals or elements of aggregate structures. 59 | 60 | These `SvIsBOOL()` values will still be subject to the usual semantics regarding macros like `SvPV` or `SvIV` - so numerically they will still be 1 or 0, and stringily they will still be "1" and "" (though see the Future Scope section below). 61 | 62 | It may be possible to find a spare `SvFLAGS` field for this purpose, at which point there would be macros on a theme similar to the `SvPOK`/`SvIOK`/etc.. set. I would suggest avoiding the single-letter abbreviations of the older style of code - letters aren't so expensive these days that we can't at least spell out the word "BOOL" in full. 63 | 64 | Flag bits being rare and expensive, it may turn out that finding a spare flag bit is simply impossible (or at least, undesired). This could be implemented instead by using special pointer values in the `SvPVX()` slots of such booleans. A small change to `sv_setsv()` and friends would be possible to cause it to copy the actual pointer value itself, such that any "boolean" SV can be distinguished by pointer equality of this field, to that of the original "yes" and "no" immortals. 65 | 66 | ### Perl-level Interface 67 | 68 | Once the interpreter can reliably store the concept of "is a boolean" and expose this at least to XS modules, the question remains on how pureperl code can make use of it. How can boolean values be created, and how can we test if a given value is a boolean? 69 | 70 | For creating such values, in many cases it is sufficient to simply use any of the existing boolean predicate operators, for example the comparison operators: 71 | 72 | ```perl 73 | my $sv = 4 > 5; # The $sv now has SvIsBOOL 74 | ``` 75 | 76 | It may be considered useful to provide two new zero-arity functions, `true` and `false`, to explicitly create these values (which are at least a little more obvious in intent than the equivalent `!!1` and `!!0`). These can be requested from the `builtin` module (PPC 0009): 77 | 78 | ```perl 79 | use builtin 'true', 'false'; 80 | 81 | func(1, "1", true); # Three distinct values 82 | func(0, "0", "", false); # Four distinct values 83 | ``` 84 | 85 | The question remains on how pureperl code can inspect the "type" of a value to inspect if it is one of these special boolean values. While perl core doesn't have any testing keywords for this, the nearest match to currently available features may be to add a new builtin function: 86 | 87 | ```perl 88 | use builtin 'isbool'; 89 | 90 | sub distinguish($x) { 91 | say !isbool $x ? "not a boolean" : 92 | $x ? "true" : 93 | "false"; 94 | } 95 | ``` 96 | 97 | Whatever solution is considered here should be designed to interact well with any possible larger considerations from the fallout of stronger string-vs-number distinctions, and other ideas currently floating around. 98 | 99 | I accept that this is the weakest part of this suggestion so far, and welcome comment here in particular (though *please* first read the "Future Scope" section below). 100 | 101 | ## Backwards Compatibility 102 | 103 | There are not expected to be any backwards compatiblity problems with this proposal. At the interpreter level, extra information is being added to certain SV values, without changing the meaning of any existing information currently stored. Code that is unaware of the new semantics will continue to see existing values unaltered. 104 | 105 | Likewise at the Perl syntax level, the only new functionality being added consists of new functions in the `builtin` namespace (PPC 0009). 106 | 107 | ## Security Implications 108 | 109 | Likewise, as this proposal only adds extra information to that being stored by SVs, it is not anticipated there are any security implications of doing so. There is not expected to be a security effect from exposing the fact that some SVs happen to express values of boolean intent. 110 | 111 | ## Examples 112 | 113 | (See embedded code above) 114 | 115 | ## Prototype Implementation 116 | 117 | I made an attempt at using `SvPVX()` tracking of the immortal constants and the `SvIsBOOL()` test macro, at 118 | 119 | [https://github.com/leonerd/perl5/tree/stable-bool](github.com/leonerd/perl5/tree/stable-bool) 120 | 121 | This was accepted and merged into core perl in time to be released as part of Perl version 5.35.4. 122 | 123 | There is no attempt yet to provide the `true`, `false` or `isbool` builtin functions as the builtin function mechanism is not yet available. A workaround for `isbool` is provided by `Scalar::Util`. 124 | 125 | ## Future Scope 126 | 127 | By its nature this proposal fits in the category of "give Perl values more typing information". As such it is likely to fit well with other ideas of a similar nature - such as clearer distinction between strings and numbers, between Unicode text and byte buffers, and other concepts. It may become possible to group these ideas together in a better way, to better solve such questions as how to add predicate-test functions in a way that is visible to Perl syntax. 128 | 129 | It may also be possible to provide a lexically-guarded feature that alters the way that boolean values are stringified. It is noted that in Perl currently, a false value stringifies to the empty string, which leads to certain difficulties in debugging output. Perhaps under the `boolean` feature, or perhaps under its own unique feature name, boolean values could stringify differently: 130 | 131 | ```perl 132 | use feature 'boolean'; 133 | 134 | my ($x, $y) = (true, false); 135 | print "X=$x Y=$y\n"; 136 | 137 | __END__ 138 | X=true Y=false 139 | ``` 140 | 141 | ## Rejected Ideas 142 | 143 | One thing that is certainly impossible to do currently is change the stringification of existing boolean values. There is far too much code around - primarily in unit tests - which relies on the current stringification of boolean true and false values, and this cannot be modified. Doing so would break such tests as: 144 | 145 | ```perl 146 | is( somefunc(), "", 'somefunc returns false' ); 147 | ``` 148 | 149 | ## Open Issues 150 | 151 | ## Copyright 152 | 153 | Copyright (C) 2021, Paul Evans. 154 | 155 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 156 | -------------------------------------------------------------------------------- /ppcs/ppc0009-builtin-namespace.md: -------------------------------------------------------------------------------- 1 | # Namespace for Builtin Functions 2 | 3 | ## Preamble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0009 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | Define a mechanism by which functions can be requested, for functions that are provided directly by the core interpreter. 13 | 14 | ## Motivation 15 | 16 | The current way used to request utility functions in the language is the `use` statement, to either request some symbols be imported from a utility module (some of which are shipped with perl core; many more available on CPAN), or to enable some as named features with the `use feature` pragma module. 17 | 18 | ```perl 19 | use List::Util qw( first any sum ); 20 | use Scalar::Util qw( refaddr reftype ); 21 | use feature qw( say fc ); 22 | ... 23 | ``` 24 | 25 | In this case, `say` and `fc` are provided by the core interpreter but everything else comes from other files loaded from disk. The `Scalar::Util` and `List::Util` modules happen to be dual-life shipped with core perl, but many others exist on CPAN. 26 | 27 | This document proposes a new mechanism for providing commonly-used functions that should be considered part of the language: 28 | 29 | ```perl 30 | use builtin qw( reftype ); 31 | say "The reference type of ref is ", reftype($ref); 32 | ``` 33 | 34 | or 35 | 36 | ```perl 37 | # on a suitably-recent perl version 38 | say "The reference type of ref is ", builtin::reftype($ref); 39 | ``` 40 | 41 | **Note:** This proposal largely concerns itself with the overal *mechanism* used to provide these functions, and expressly does not go into a full detailed list of individual proposed functions that ought to be provided. A short list is given containing a few likely candidates to start with, in order to experiment with the overall idea. Once found stable, it is expected that more functions can be added on a case-by-case basis; perhaps by using the PPC process or not, as individual cases require. In any case, it is anticipated that this list would be maintained on an ongoing basis as the language continues to evolve, with more functions being added at every release. 42 | 43 | ## Rationale 44 | 45 | While true syntax additions such as infix operators or control-flow keywords have often been added to perl over the years, there has been little advancement in regular functions - or at least, operators that appear to work as regular functions. Where they have been added, the `use feature` mechanism has been used to enable them. This has two notable downsides: 46 | 47 | * It confuses users, by conflating the control-flow or true-syntax named features (such as `try` or `postderef`), with ones that simply add keywords that look and feel like regular functions (such as `fc`). 48 | 49 | * Because `feature` is a core-shipped module that is part of the interpreter it cannot be dual-life shipped to CPAN, so newly-added functions cannot easily be provided to older perls. 50 | 51 | As there have not been many new regular functions added to the core language itself, the preferred mechanism thus far has been to add functions to core-shipped dual-life modules such as [Scalar::Util](https://metacpan.org/pod/Scalar::Util). This itself is not without its downsides. Although it is possible to import functions lexically, almost no modules do this; instead opting on a package-level import into the caller. This has the effect of making every imported utility function visible from the caller's namespace - which can be especially problematic if the caller is attempting to provide a class with methods: 52 | 53 | ```perl 54 | package A::Class { 55 | use List::Util 'sum'; 56 | ... 57 | } 58 | 59 | say A::Class->new->sum(1, 2, 3); # inadvertantly visible method 60 | 61 | # This will result in some large number being printed, because the 62 | # List::Util::sum() function will be invoked with four arguments - the 63 | # numerical value of the new instance reference, in addition to the three 64 | # small integers given. 65 | 66 | ``` 67 | 68 | A related issue here is that the process of adding new named operators to the perl language is very involved and requires a lot of steps - many updates to the lexer, parser, core list of opcodes, and so on. This creates a high barrier-to-entry for any would-be implementors who wish to provide a new regular function. 69 | 70 | ## Specification 71 | 72 | This document proposes an implementation formed from two main components as part of core perl: 73 | 74 | 1. A new package namespace, `builtin::`, which is always available in the `perl` interpreter (and thus acts in a similar fashion to the existing `CORE::` and `Internals::` namespaces). 75 | 76 | 2. A new pragma module, `use builtin`, which lexically imports functions from the builtin namespace into its calling scope. 77 | 78 | As named pragma modules are currently implemented by the same file import mechanism as regular modules, this necessitates the creation of a `builtin.pm` file to contain at least part of the implementation - perhaps the `&builtin::import` function itself. This being the case, a third component can be created: 79 | 80 | 3. A new module `builtin.pm` which can be dual-life shipped as a CPAN distribution called `builtin`, to additionally contain an XS implementation of the provided functions. 81 | 82 | This combination of components has the following properties: 83 | 84 | * By use of the new package namespace, code written for a sufficiently-new version of perl can already make use of the provided functions by their fully-qualified name: 85 | 86 | ```perl 87 | say "The reference type of anonymous arrays is ", builtin::reftype([]); 88 | ``` 89 | 90 | * Code can be written which imports these functions to act as regular named functions, in a similar way familiar from utility modules like `Scalar::Util`: 91 | 92 | ```perl 93 | use builtin 'reftype'; 94 | say "The reference type of anonymous arrays is ", reftype([]); 95 | ``` 96 | 97 | * Named functions are imported lexically into the calling block, not symbolically into the calling package. This does differ from the behaviour of traditional utility function modules, but more closely matches the expectations from other pragma modules such as `feature`, `strict` and `warnings`. Overall it is felt this is justified by its lowercase name suggesting its status as a special pragma module. 98 | 99 | * Object classes do not inadvertantly expose them all as named methods: 100 | 101 | ```perl 102 | package A::Class { 103 | use builtin 'sum'; 104 | ... 105 | } 106 | 107 | say A::Class->new->sum(1, 2, 3); # results in the usual "method not found" exception behaviour 108 | ``` 109 | 110 | Although this document does not wish to fully debate the set of functions actually provided, some initial set is required in order to bootstrap the process and experiment with the mechanism. Rather than proposing any new functions with unclear design, I would recommend sticking simply to copying existing widely-used functions that already ship with core perl in utility modules: 111 | 112 | * From `Scalar::Util`, copy `blessed`, `refaddr`, `reftype`, `weaken`, `isweak`. 113 | 114 | * From `Internals`, copy `getcwd` (because it is used by some core unit tests, and it would be nice to remove it from the `Internals` namespace where it ought never have been in the first place). 115 | 116 | ## Backwards Compatibility 117 | 118 | This proposal does not introduce any new syntax or behavioural change, aside from a new namespace for functions and a new pragma module. As previous perl versions do not have a `builtin::` namespace nor a `use builtin` pragma module, no existing code will be written expecting to make use of them. Thus there is not expected to be any compability concerns. 119 | 120 | As a related note, by creating a dual-life distribution containing the `builtin.pm` pragma module along with a polyfill implementation of any functions it ought to contain, this can be shipped to CPAN in order to allow code written using this new mechanism to be at least partly supported by older perl versions. Because the pragma still works as a regular module, code written using the `use builtin ...` syntax would work as intended on older versions of perl if the dual-life `builtin` distribution is installed. 121 | 122 | ## Security Implications 123 | 124 | There are none anticipated security implications of the builtin function mechanism itself. However, individual functions that are added will have to be considered individually. 125 | 126 | ## Examples 127 | 128 | ## Prototype Implementation 129 | 130 | None yet. 131 | 132 | ## Future Scope 133 | 134 | As this proposal does not go into a full list of what specific functions might be provided by the mechanism, this is the main area to address in future. As a suggestion, I would make the following comments: 135 | 136 | * Most of the `Scalar::Util` functions should be candidates 137 | 138 | * Most of the `List::Util` functions that do not act as higher-order functionals can probably be included. This would be functions like `sum`, `max`, `pairs`, `uniq`, `head`, etc. The higher-order functionals such as `reduce` or its specialisations like `first` and `any` would not be candidates, because of their "block-like function as first argument" parsing behaviour at compiletime. 139 | 140 | * Some of the `POSIX` functions that act abstractly as in-memory data utilities, such as `ceil` and `floor`. I would not recommend adding the bulk of the operating system interaction functions from POSIX. 141 | 142 | * There are other PPCs or Pre-PPC discussions that suggest adding new named functions that would be good candidates for this module. They can be considered on their own merit, by reference to this PPC. At time of writing this may include new functions to handle core-supported boolean types (PPC 0008) or the new module-loading function (PPC 0006). 143 | 144 | * Once a stable set of functions is defined, consider creating version-numbered bundles in a similar theme to those provided by `feature.pm`: 145 | 146 | ```perl 147 | use builtin ':5.40'; # imports all those functions defined by perl v5.40 148 | ``` 149 | 150 | * Once version-numbered bundles exist, consider whether the main `use VERSION` syntax should also enable them; i.e. 151 | 152 | ```perl 153 | use v5.40; # Does this imply use builtin ':5.40'; ? 154 | ``` 155 | 156 | ## Rejected Ideas 157 | 158 | ### Multiple Namespaces 159 | 160 | An initial discussion had been to consider giving multiple namespaces for these functions to live in, such as `string::` or `ref::`. That was eventually rejected as being overly complex, and inviting a huge number of new functions. By sticking to a single namespace for all regular functions, we apply a certain amount of constraining pressure to limit the number of such functions that are provided. 161 | 162 | ## Open Issues 163 | 164 | ### Package Name 165 | 166 | What is the package name these functions are provided in? 167 | 168 | The discussion above used `builtin::`. Other proposed suggestions include `function::` or `std::`. 169 | 170 | ### Pragma Module Name 171 | 172 | What is the module name for the lexical-import pragma? 173 | 174 | The discussion above used `use builtin`, to match the package name. Technically it does not have to match the package name. In particular, if during implementation or initial use it is found to be problematic that the name does match, the import module could use a plural form of the same word; as 175 | 176 | ```perl 177 | use builtins qw( function names here ); 178 | ``` 179 | 180 | ### Version Numbering 181 | 182 | How to version number the `builtin` pragma module? 183 | 184 | This becomes an especially interesting when considering the dual-life module distribution provided as a polyfill for older perls. 185 | 186 | A case can be made that its version number should match the version of perl itself for which it provides polyfill functions. Thus, code could write: 187 | 188 | ```perl 189 | use builtin v5.40; 190 | # and now all the builtin:: functions from perl v5.40 are available 191 | ``` 192 | 193 | This does initially seem attractive, until one considers the possibility that a dual-life implementation of these polyfills might contain bugs that require later revisions to fix. How would the version numbering of the dual-life distribution reflect the fact that the implementation contains a bugfix on top of these? 194 | 195 | ### Polyfill for Unavailable Semantics 196 | 197 | While not directly related to the question of how to provide builtin functions to new perls, by offering to provide a dual-life module on CPAN as a polyfill for older perl releases, the question arises on what to do if older perls cannot support the semantics of a provided function. The current suggestion of copying existing functions out of places like `Scalar::Util` does not cause this problem, but when we consider some of the additional PPCs we run into some more complex edge-cases. 198 | 199 | For example, PPC 0008 proposes adding new functions `true` and `false` to provide real language-level boolean values, and an `isbool` predicate function to enquire whether a given value has boolean intention. The first two can be easily provided on older perls, but polyfilling this latter function is not possible, because the question of "does this value have boolean intention?" is not a meaningful question to ask on such perls. There are a number of possible ways to handle this situation: 200 | 201 | 1. Refuse to import the symbol - `use builtin 'isbool'` would fail at compiletime 202 | 203 | 2. Import, but refuse to invoke the function - `if( isbool $x ) { ... }` would throw an exception 204 | 205 | 3. Give a meaningful but inaccurate answer - `isbool $x` would always return false, as the concept of "boolean intention" does not exist here 206 | 207 | Each of these could be argued as the correct behaviour. While it is not directly a question this PPC needs to answer, it is at least acknowledged that some added polyfill functions would have this question, and it would be encouraged that all polyfilled functions should attempt to act as consistently as reasonably possible in this regard. 208 | 209 | ## Copyright 210 | 211 | Copyright (C) 2021, Paul Evans. 212 | 213 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 214 | -------------------------------------------------------------------------------- /ppcs/ppc0011-slurp-argument.md: -------------------------------------------------------------------------------- 1 | # Command-line flag for slurping 2 | 3 | ## Preamble 4 | 5 | Author: Tomasz Konojacki 6 | Sponsor: 7 | ID: 0011 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | Introduce a new command-line flag for `perl`, `-g`, which sets `$/` to `undef`, 13 | and thus enables slurp mode. It is a simpler alias for `-0777`. 14 | 15 | ## Motivation 16 | 17 | Slurping (i.e. reading a whole file at once, instead of line by line) is a very 18 | common operation in one-liners, and therefore it deserves its own dedicated 19 | flag. 20 | 21 | ## Rationale 22 | 23 | Currently, `-0777` is the most common way to enable slurp mode in one-liners. 24 | It's a special case of `-0number`. When `number` is above `0o377` it sets `$/` 25 | to `undef`, which enables slurp mode. 26 | 27 | `-0number` suffers from the following problems: 28 | 29 | - The input record separator has to be specified with an octal number, which is 30 | very unusual. 31 | 32 | - It's overly general, it can set `$/` to any character. Users rarely need 33 | values other than `undef` or `"\n"`. 34 | 35 | - Its most common use, enabling slurp mode, is a special case hidden behind 36 | a magic number. 37 | 38 | A dedicated flag for slurping would allow users to avoid the peculiarities of 39 | `-0number`. 40 | 41 | ## Specification 42 | 43 | `-g` is an alias for `-0777`, they are completely equivalent. 44 | 45 | ## Backwards Compatibility 46 | 47 | No breakage is expected. `perl -g` is currently a fatal error. 48 | 49 | ## Security Implications 50 | 51 | Hopefully none. 52 | 53 | ## Examples 54 | 55 | # collapse consecutive newlines: 56 | perl -i -gpE 's/\n+/\n/g' file.txt 57 | 58 | ## Future Scope 59 | 60 | ## Rejected Ideas 61 | 62 | - Long flag, e.g. `--slurp`. Perl currently doesn't support long flags and 63 | adding them would be beyond the scope of this PPC. 64 | 65 | - Alternative spellings of the flag, e.g. `-R`, `-o`. The author believes none 66 | of them are better or worse than `-g`. Unfortunately, the most natural choice, 67 | `-s`, [is already taken](https://perldoc.perl.org/5.34.0/perlrun#-s). 68 | 69 | ## Open Issues 70 | 71 | ## Copyright 72 | 73 | Copyright (C) 2021 Tomasz Konojacki 74 | 75 | This document and code and documentation within it may be used, redistributed 76 | and/or modified under the same terms as Perl itself. 77 | -------------------------------------------------------------------------------- /ppcs/ppc0012-configure-no-taint.md: -------------------------------------------------------------------------------- 1 | # Configure option for not including taint support 2 | 3 | ## Preamble 4 | 5 | Author: Neil Bowers 6 | Sponsor: 7 | ID: 0012 8 | Status: Accepted 9 | 10 | 11 | ## Abstract 12 | 13 | Add a Configure option for deciding whether to include taint support 14 | in this build of Perl. 15 | This is essentially a Configure wrapper 16 | around the existing `-DSILENT_NO_TAINT_SUPPORT` compile flag. 17 | The question would be worded as 18 | 19 | > "Do you want to build Perl with taint support? [y]", 20 | 21 | so the default of "Yes" means that Perl will build as previously. 22 | 23 | ## Motivation 24 | 25 | * Support for taint adds a runtime overhead of roughly 10%. 26 | * Adding a Configure option makes it easier for people to build a 27 | Perl without taint support. 28 | * Many people don't even know there's an option to build Perl without 29 | taint support, or what the benefit is. 30 | This will improve awareness. 31 | * The default of "yes" means that if people aren't paying attention 32 | when running Configure, then they'll get a perl similar to the last 33 | one they configured, and no code will unexpectedly break. 34 | * Having this as a Configure option means you'll then be able to 35 | query %Config to determine whether your 36 | perl was built with taint support. 37 | 38 | ## Rationale 39 | 40 | * Anecdotally, a very high percentage of Perl developers never use 41 | the taint features, yet we're all taking the performance hit. 42 | * We want the default to be that Perl continues to be built with 43 | taint support, for backwards compatibility. 44 | * An easier question would be "Do you want taint support?", 45 | but my assumption is that Configure options should default to "No", 46 | which is why this is proposing a slightly more clumsy question. 47 | 48 | ## Specification 49 | 50 | ### What do we mean by "no Taint support"? 51 | 52 | There are currently two compilation options: 53 | 54 | 1. `SILENT_NO_TAINT_SUPPORT`: no taint support, but you can still call `-T`, 55 | it just won't do anything. 56 | 2. `NO_TAINT_SUPPORT`: no taint support, trying to call `-T` is fatal. 57 | 58 | Following discussion on the initial proposal, 59 | we now propose that the Configure option sets `-DSILENT_NO_TAINT_SUPPORT`. 60 | There are thousands of distributions on CPAN with tests that start with 61 | 62 | #!perl -T 63 | 64 | Almost all of these don't require the -T option, 65 | but if we make it fatal, then people won't be able to install a large chunk of CPAN. 66 | 67 | Once this is implemented, we'll be able to identify which CPAN distributions fail 68 | to install with taint not enabled, and then decide whether they need updating. 69 | 70 | ### Configure preamble 71 | 72 | Before the specific question is asked, 73 | Configure should present an explanation. 74 | 75 | > Perl can provide a set of special security checks, 76 | > which are known as *taint mode*. 77 | > The most well-known of these is that data derived from 78 | > outside your program should not be trusted ("is tainted") 79 | > until you have checked it. 80 | > 81 | > These days there are many more security considerations, 82 | > and as a result taint mode isn't widely used. 83 | > But support for it adds a runtime overhead, 84 | > whether or not you use it. 85 | > As a result, you can choose to build Perl without taint support. 86 | 87 | ### Configure prompt 88 | 89 | > "Do you want to build Perl with taint support? [y]", 90 | 91 | ### Configure variable 92 | 93 | The name proposed for the Configure variable is `taint_supported`. 94 | Because it won't exist in older versions of Perl, 95 | you'd have to check this with logic like this: 96 | 97 | use Config; 98 | if (!exists($Config{taint_supported}) || $Config{taint_supported}) { 99 | # This perl supports taint 100 | } 101 | else { 102 | # This perl does NOT support taint 103 | } 104 | 105 | 106 | ### Documentation changes 107 | 108 | **perlsec** will need to be updated, to explain that the availability 109 | of these features depends on how Perl was configured at build time. 110 | 111 | Should the documentation be modified at Configure time, 112 | to reflect the builder's decision? 113 | I don't know what precedents there are for this? 114 | 115 | **perlrun** will have to be updated, 116 | for example to say that `-T` relies on Perl having been compiled 117 | with taint support. 118 | 119 | All other documentation that references taint mode will need to be 120 | reviewed to decide whether they should be updated. 121 | 122 | ## Backwards Compatibility 123 | 124 | There have been compile flags, `-DNO_TAINT_SUPPORT` and `-DSILENT_NO_TAINT_SUPPORT`, 125 | for some years, so we're not really changing much here, 126 | but adding this to Configure makes people aware that this option exists. 127 | 128 | If you Configure Perl with -d, you'll still get taint support. 129 | 130 | 131 | ## Security Implications 132 | 133 | Not applicable, since we're just putting a thin layer on top 134 | of an existing capability. 135 | 136 | Famous last words. 137 | 138 | ## Examples 139 | 140 | None. 141 | 142 | ## Prototype Implementation 143 | 144 | None. 145 | 146 | ## Future Scope 147 | 148 | We might consider switching the default to be "don't include taint support" 149 | at some point in the future, 150 | but before we consider this, 151 | we need a lot more testing without taint support, 152 | for example to see what the result is on CPAN. 153 | 154 | ## Rejected Ideas 155 | 156 | 1. **Do Nothing**. Many people on p5p weren't aware of the existing capability 157 | (the compile flag), but on learning of its existence were keen to use it. 158 | Extrapolating, the author of this PPC believes that a significant 159 | percentage of Perl developers / users would happily make the trade-off 160 | (of no taint support, for improved performance), if they had the option. 161 | 2. **Default to non-taint support**. I think people would rightly be surprised 162 | if the new Configure option defaulted to "no taint support". 163 | Anyone currently using taint support would be surprised if they ran 164 | `Configure -des` and then found that their taint-using code stopped working. 165 | 3. Initially I thought this should set `-DNO_TAINT_SUPPORT`, so use of -T would be fatal. 166 | So if someone built Perl without taint support, they'd want to know if something 167 | wasn't working because of that change. But it would also mean they couldn't install 168 | many CPAN distributions, and it's not realistic to expect them to work around that. 169 | 4. Initially I suggested the question should be "Do you want to drop support for taint? [n]", 170 | so that the new option could default to "No". But on discussing it with Tux, he felt 171 | that people would be confused by the double negative, so we agreed to switch the sense 172 | of the question. 173 | 174 | ## Open Issues 175 | 176 | * The exact wording of the Configure preamble 177 | * The exact wording of the Configure question 178 | * The name of the Configure variable 179 | * Should perlsec be modified depending on the Configure decision, 180 | or be static? 181 | * `SILENT_NO_TAINT_SUPPORT` or `NO_TAINT_SUPPORT`? 182 | * Should the sense of the question be "exclude" or "include"? 183 | Include is easier for people to understand, but results in a default of "y". 184 | 185 | I think these question have been resolved far enough here to make a decision on the PPC, 186 | and the final details of prompting etc can be left to the implementation. 187 | 188 | ## Copyright 189 | 190 | Copyright (C) 2022, Neil Bowers. 191 | 192 | This document and code and documentation within it may be used, 193 | redistributed and/or modified under the same terms as Perl itself. 194 | 195 | -------------------------------------------------------------------------------- /ppcs/ppc0013-overload-join-and-substr.md: -------------------------------------------------------------------------------- 1 | # Support overloaded objects in join(), substr() builtins 2 | 3 | ## Preamble 4 | 5 | Authors: Eric Herman , Philippe Bruhat 6 | Sponsor: Paul Evans 7 | ID: 0013 8 | Status: Accepted 9 | 10 | ## Abstract 11 | 12 | As of Perl version 5.38, `overload` is incomplete and surprising for 13 | string operations: 14 | 15 | * the `join` builtin should be using the `concat` (`.`) overload 16 | when operating on overloaded objects (including the separator), 17 | * `overload` should support the `substr` operation. 18 | 19 | ## Motivation 20 | 21 | The perl builtin `join`, when operating on overloaded objects will 22 | stringify them and the result will no longer be an object of the class 23 | with overloads, even if the `concat` (`.`) function is overloaded. 24 | 25 | Additionally, the `overload` module does not currently support overloading 26 | the `substr` operation. 27 | 28 | The current behaviour is inconsistent and confusing. 29 | 30 | There is at least one CPAN module which works around the `join` 31 | deficiency, (the [`join` function of 32 | `String::Tagged`](https://metacpan.org/pod/String::Tagged#join), but not 33 | universally. 34 | 35 | The CPAN module 36 | [`overload::substr`](https://metacpan.org/pod/overload::substr)) is a 37 | complete solution, but we think that deferring to CPAN when Perl should 38 | do the right thing is not a satisfying answer. 39 | 40 | ## Rationale 41 | 42 | Extending `overload` to support `substr`, and `join` to use `concat` 43 | would be less surprising and more consistent with the "Do What I Mean" 44 | spirit of Perl. 45 | 46 | Adding this to Perl would make `overload::substr` obsolete and simplify 47 | the implementation of `String::Tagged` and other similar modules which 48 | have written their own `join`. 49 | 50 | ## Specification 51 | 52 | The documentation for `join` would be amended with the following: 53 | 54 | > If any of the arguments (`EXPR` or `LIST` contents) are objects which 55 | > overload the concat (`.`) operation, then that overloaded operation 56 | > will be invoked. 57 | 58 | The documentation for `overload` would be amended to extend the complete 59 | list of keys that can be specified in the `use overload` statement to 60 | include `substr`. The module `overload::substr` code and documentation 61 | should be used as a reference and guide for implementation in core. Note 62 | that the current documentation for this module highlights a need for 63 | additional tests, which must be a part of a core implementation. 64 | 65 | An open question is whether `split` requires an overload target, 66 | and should a fallback implementation be provided using the 67 | overloaded `substr`? 68 | 69 | ## Backwards Compatibility 70 | 71 | ### `substr` 72 | 73 | There is no need autogenerate an implementation for `substr` if it is 74 | missing, as the stringification will happen as it does now. 75 | 76 | `overload::substr` would become an optional dependency for modules 77 | that needs to support older Perls. 78 | 79 | ### `join` 80 | 81 | If code relies upon the `join` function to convert objects to plain 82 | strings, this would break that code, thus a feature flag seems 83 | necessary. 84 | 85 | Modules with overloading, which had to implement their own `join` (to 86 | get a different behaviour than the stringification of the builtin 87 | `join`) could add version-conditional logic to work with different 88 | versions of Perl and delegate to the builtin `join` if the feature is 89 | available. New modules may choose not to implement their own `join`, 90 | thus dropping support for older versions of perl. 91 | 92 | ## Security Implications 93 | 94 | We do not yet foresee security issues. Guidance is welcome. 95 | 96 | ## Examples 97 | 98 | For `join`: 99 | 100 | ```perl 101 | # When @list contains elements with concat overloading, we expect this code: 102 | 103 | my $ret = join $sep, @list; 104 | 105 | # To behave like this code: 106 | 107 | use List::Util 'reduce'; 108 | 109 | my $ret = reduce { ( $a . $sep ) . $b } @list; 110 | ``` 111 | 112 | For `substr`, the `overload::substr` module will be the guide for an 113 | initial implementation. Its documentation includes the following example: 114 | 115 | ```perl 116 | package My::Stringlike::Object; 117 | 118 | use overload::substr; 119 | 120 | sub _substr 121 | { 122 | my $self = shift; 123 | if( @_ > 2 ) { 124 | $self->replace_substr( @_ ); 125 | } 126 | else { 127 | return $self->get_substr( @_ ); 128 | } 129 | } 130 | ``` 131 | 132 | ## Prototype Implementation 133 | 134 | For `substr`: . 135 | 136 | ## Future Scope 137 | 138 | Looking at `perlfunc`, we find numerous builtins which, when applied 139 | on a string, will return a modified version of the string. Each of these 140 | is worthy of discussion: 141 | 142 | * `chomp` 143 | * `chop` 144 | * `fc` 145 | * `lc` 146 | * `lcfirst` 147 | * `quotemeta` 148 | * `reverse` 149 | * `split` 150 | * `tr///` and `y///` 151 | * `uc` 152 | * `ucfirst` 153 | 154 | ## Rejected Ideas 155 | 156 | Not applicable. 157 | 158 | ## Open Issues 159 | 160 | None yet. 161 | 162 | ## Copyright 163 | 164 | Copyright (C) 2021, Philippe Bruhat and Eric Herman. 165 | 166 | This document and code and documentation within it may be used, 167 | redistributed and/or modified under the same terms as Perl itself. 168 | -------------------------------------------------------------------------------- /ppcs/ppc0014-english-aliases.md: -------------------------------------------------------------------------------- 1 | # `${^ENGLISH_NAME}` aliases for punctuation variables 2 | 3 | ## Preamble 4 | 5 | Author: Graham Knop 6 | ID: 0014 7 | Status: Draft 8 | 9 | ## Abstract 10 | 11 | Provide aliases for all punctuation variables with the form 12 | `${^ENGLISH_NAME}`, with the english name taken from the aliases currently 13 | provided by `English.pm`. 14 | 15 | ## Motivation 16 | 17 | Punctuation variables are not very friendly to use. Many people consider them 18 | ugly, and they require you to memorize all of the relevant variables. It would 19 | be valuable to provide better names for these variables as part of the core 20 | language. `English.pm` does exist as an attempt at this but brings its own 21 | issues. The English names look like `$ENGLISH_NAME`, which is syntactically 22 | the same as a package variable that a user might define themselves. While not 23 | exactly rare, use of `English.pm` is not particularly common either. Many 24 | seasoned Perl programmers would not know the English names for many commonly 25 | used punctuation variables without consulting a reference. 26 | 27 | ## Rationale 28 | 29 | Some of the newer superglobals are spelled like `${^ENGLISH_NAME}`. Providing 30 | aliases to all of the punctuation variables would allow them to be more self 31 | documenting, like the `English.pm` names, but without the being syntactically 32 | identical to a normal package variable. Being spelled like `${^ENGLISH_NAME}` 33 | makes it obvious that this is a special variable. This makes the magic 34 | behavior more obvious, and helps point to the correct reference material. 35 | Since there are already several "punctuation" variables with this spelling, it 36 | should be familiar to Perl programmers. It also means the variables are always 37 | available without needing to load an additional module. 38 | 39 | The names used by `English.pm` could mostly be re-used with the new spelling. 40 | This should help any Perl programmer that has preferred `English.pm` to 41 | understand the new form. 42 | 43 | ## Specification 44 | 45 | For each variable listed in perlvar which includes an `English.pm` name like 46 | `$OS_ERROR`, provide an alias named `${^OS_ERROR}` as part of the core 47 | language. 48 | 49 | ## Backwards Compatibility 50 | 51 | There won't be any conflict regarding syntax, as this is already valid syntax, 52 | and already referring to super-globals. This does present an issue, as 53 | variables with this syntax are exempt from strict. Attempting to use these 54 | variables will fail silently when run on older perls, or when misspelling the 55 | variable. 56 | 57 | A shim could be created to allow using these variables on older perls. It may 58 | not be able to be 100% compatible. 59 | 60 | ## Security Implications 61 | 62 | Many of the punctuation variables relate to system state or errors. 63 | Misspelling a variable like `${^EVAL_ERROR}` (aka `$@`) could lead to an error 64 | being ignored rather than handled, which could lead to security concerns. 65 | 66 | ## Examples 67 | 68 | ```perl 69 | local $_; 70 | local ${^ARG}; 71 | 72 | my $pid = $$; 73 | my $pid = ${^PID}; 74 | 75 | my $error = $@; 76 | my $error = ${^EVAL_ERROR}; 77 | 78 | my %signals = %SIG; 79 | my %signals = %{^SIG}; 80 | ``` 81 | 82 | ## Prototype Implementation 83 | 84 | An initial attempt at a backwards compatibility shim: 85 | https://github.com/haarg/English-Globals/blob/master/lib/English/Globals.pm 86 | 87 | A more complete implementation written in XS: 88 | https://metacpan.org/pod/English::Name 89 | 90 | Another implementation of the same idea from 2017, in pure perl: 91 | https://metacpan.org/pod/English::Control 92 | 93 | ## Future Scope 94 | 95 | Most punctuation characters are already used for special variables, so a typo 96 | can't really be caught. With english names, there is a greater risk of typos, 97 | but also a greater possibility of catching them. Possibly variables with this 98 | format could have strict applied to them. 99 | 100 | Providing these aliases could make it possible to deprecate and remove some 101 | lesser-used punctuation variables, freeing them up for use as syntax rather 102 | than variables. This was previously done with the `$*` variable. 103 | 104 | ## Rejected Ideas 105 | 106 | `English.pm` serves as a previous example of trying to make punctuation 107 | variable use more friendly, but has various caveats as covered in the 108 | "Motivation" section. 109 | 110 | Spelling these variables like `$^ENGLISH_NAME` is not possible because 111 | variables of the form `$^E` exist. When interpolating, `"$^OS_ERROR"` is 112 | parsed as `"${^O}S_ERROR"`. If this brace-less form was still desired, this 113 | PPC would still be a prerequisite. When interpolating, a delimited form is 114 | required to allow word characters to follow the variable. This delimited form 115 | would be the same `${^ENGLISH_NAME}` form proposed by this PPC. 116 | 117 | `$*ENGLISH_NAME` has been proposed, but conflicts with glob dereferencing. 118 | Outside string interpolation, that variable syntax is currently invalid and 119 | thus could be used. But when interpolating, a delimited form is needed. This 120 | would be `${*ENGLISH_NAME}`, but that syntax is already valid, as a scalar 121 | dereference of a glob. Trying to use `$*{ENGLISH_NAME}` is also a problem 122 | because it is currently valid syntax, refering to an element in the `%*` hash. 123 | 124 | ## Open Issues 125 | 126 | - None at this time 127 | 128 | ## Copyright 129 | 130 | Copyright (C) 2022, Graham Knop. 131 | 132 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 133 | -------------------------------------------------------------------------------- /ppcs/ppc0015-drop-old-package-separator.md: -------------------------------------------------------------------------------- 1 | # Remove apostrophe ("Old package separator") as package separator 2 | 3 | ## Preamble 4 | 5 | Author: Nicolás Mendoza 6 | Sponsor: 7 | ID: 0015 8 | Status: Draft 9 | 10 | ## Abstract 11 | 12 | Remove support for ' (apostrophe) as package namespace separator. 13 | 14 | ## Motivation 15 | 16 | * Removal of ambiguous syntax that complicates parsing and confuses users. 17 | 18 | From perldoc perlmod 19 | > The old package delimiter was a single quote, but double colon is now the preferred delimiter, in part because it's more readable to humans, and in part because it's more readable to emacs macros. It also makes C++ programmers feel like they know what's going on--as opposed to using the single quote as separator, which was there to make Ada programmers feel like they knew what was going on. Because the old-fashioned syntax is still supported for backwards compatibility, if you try to use a string like `"This is $owner's house"`, you'll be accessing `$owner::s`; that is, the `$s` variable in package owner, which is probably not what you meant. Use braces to disambiguate, as in `"This is ${owner}'s house"`. 20 | 21 | ## Rationale 22 | 23 | * We have been warning against its usage for decades. 24 | * It was there merely for easier adoption, but has been used very little. 25 | 26 | ## Alternatives 27 | 28 | There are a few alternatives on how to disable this feature and changing the meaning of `'` 29 | 30 | ### String interpolation of identifiers: 31 | 32 | * 1\. Treat `'` as text — `"$isn't" eq "${isn}'t"` and `"error: '$msg'" eq "error: '${msg}'"` 33 | * 1w. same with warning 34 | 35 | * 2\. Treat `'` as part of identifier — `"$isn't" eq $isn't ne $isn::t` and `"error: '$msg'" ne "error: '${msg}'"` 36 | * 2w. same with warning 37 | 38 | * 3\. Syntax error — existing code supports warning, could die instead, see `toke.c` and look for `tick_warn` 39 | 40 | * 4\. Current behaviour with warning — `"$isn't" eq "${isn::t}"` (warning) and `"error: '$msg'" eq "error: '${msg}'"` (no warning) 41 | 42 | ### Identifiers elsewhere (variables, function declarations etc.) 43 | 44 | * a\. Treat `'` as part of identifier — `isn't()` would still work, but would need to be declared as `sub isn't { }` instead of `package isn; sub t {}`. 45 | 46 | * b\. Syntax error 47 | 48 | * c\. Current behaviour with warning 49 | 50 | ### Discussion 51 | 52 | All of the above may be turned on or off using compiler flags, pragmas or extending warning::syntax 53 | 54 | The current patch that I made some years ago is mostly like: `1 + b` but cold turkey removing as much as possible without any flags or switches. 55 | 56 | I think `2 + a` would be an interesting approach to keep backwards-compatibility somewhat. 57 | 58 | Both `1` and `2` changes interpolation behaviour, but we have been warning about it since 5.28: https://github.com/Perl/perl5/commit/2cb35ee012cfe486aa75a422e7bb3cb18ff51336 59 | 60 | `1w`, `2w`, `3` and `4` requires to keep around `tick_warn` in toke.c 61 | 62 | `c` requires adding new code to warn correctly and keeping around ident parsing code: 63 | 64 | ``` 65 | $ perl -wlE 'sub foo'"'"'bar { "lol"; }; print foo'"'"'bar();' 66 | lol 67 | ``` 68 | 69 | ## Specification 70 | 71 | The feature should be rolled out in two steps, in two consecutive Perl stable releases 72 | 73 | 1. Add warning code to apostrophe usage in identifiers (outside of string interpolation) (alternative `4`) (as has been done for string interpolation already (alternative `c`)) [6] 74 | 75 | 2. Syntax error when using apostrophe as part of identifiers outside strings (alternative `b`), treat apostrophes inside strings as not a part of a variable when interpolating (alternative `1`) 76 | 77 | ## Backwards Compatibility 78 | 79 | This change will break backwards compatibility in two stable releases 80 | 81 | ### Known core / dual-life modules utilizing apostrophe as separator 82 | 83 | The following issues are slowly being sorted out independently of this PPC. 84 | 85 | * ~~cpan/Term-Cap `isn't` in test [trivial to replace] https://github.com/jonathanstowe/Term-Cap/pull/13~~ [fixed] 86 | * ~~dist/PathTools `isn't` in test [trivial to replace] https://github.com/Perl/perl5/pull/19865~~ [fixed] 87 | * ~~cpan/autodie test in Klingon [not THAT trivial to replace, don't know Klingon] https://github.com/pjf/autodie/issues/115~~ [fixed] 88 | 89 | The following issues might need to synchronize with the perl releases by checking versions or feature flags 90 | 91 | * ~~cpan/Test-Simple `isn::t` [would work wth 2 + b with small change, or state compatibility in docs]~~ [fixed] https://github.com/Test-More/test-more/commit/d619c1b0486422ac86b6c870241bf6138a041a8f 92 | * cpan/Scalar-List-Utils `set_subname` implementation [need to check perl version in .xs code etc] 93 | 94 | ## References 95 | 96 | * [1] Old package separator syntax #16270 (rt.perl.org#132485) https://github.com/Perl/perl5/issues/16270 97 | * [2] Discussion from 2009 at perl5-porters: 98 | * [3] Discussion from 2017 at perl5-porters: 99 | * [4] Discussion from 2021 at perl5-porters: 100 | * [5] Simple grep of `isn't` usage on CPAN: `https://grep.metacpan.org/search?q=%5E%5Cs*isn%27t%7Cisn%27t%5Cs*%5B%28%24%40%25%5D&qd=&qft=*.t` (Note that there quite a few false positives, so I'd guess more like 30 pkgs) 101 | * [6] Father Chrysostomos added a warning when used apostrophes are used inside string within a variable name: https://github.com/Perl/perl5/commit/2cb35ee012cfe486aa75a422e7bb3cb18ff51336 102 | 103 | ## Conditions to test 104 | 105 | * from @epa https://github.com/Perl/perl5/issues/16270#issuecomment-544092952 106 | 107 | ``` 108 | … 109 | 110 | Moreover, the need to parse it as a package separator sometimes affects its use as a string delimiter. 111 | 112 | sub foo { say $_[0] } 113 | foo"hi"; # works 114 | foo'hi'; # doesn't work 115 | 116 | foox"hi"; # String found where operator expected 117 | foo'"hi"; # Bad name after foo' (huh?) 118 | 119 | It's not that hot as a synonym for :​: either​: 120 | 121 | sub foo' {} # Illegal declaration of subroutine main​::foo 122 | sub foo​:: {} # OK 123 | 124 | sub 'foo { say $_[0] } 125 | 'foo 5; # Can't find string terminator "'" 126 | :​:foo 5; # OK 127 | ``` 128 | 129 | ## Copyright 130 | 131 | Copyright (C) 2022, Nicolas Mendoza 132 | 133 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 134 | -------------------------------------------------------------------------------- /ppcs/ppc0016-indexed-builtin.md: -------------------------------------------------------------------------------- 1 | # A built-in for getting index-and-value pairs from a list 2 | 3 | ## Preamble 4 | 5 | Author: Ricardo Signes 6 | Sponsor: 7 | ID: 0016 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | This PPC proposes `indexed`, a new builtin for interleaving a list of values 13 | with their index in that list. Among other things, this makes key/value 14 | iteration on arrays easy. 15 | 16 | ## Motivation 17 | 18 | With v5.36.0 poised to add n-at-a-time foreach, easily getting a list of 19 | index/value pairs from an array makes iteration over the pairs also becomes 20 | easy. 21 | 22 | ## Rationale 23 | 24 | If we start with the specific case of iterating over the indexes and values of 25 | an array using two-target foreach, we might write this: 26 | 27 | ```perl 28 | for my ($i, $value) (%array[ keys @array ]) { 29 | say "$i == $value"; 30 | } 31 | ``` 32 | 33 | This is tolerable, but a bit verbose. If we bury our target array deep in a 34 | structure, we get this: 35 | 36 | ```perl 37 | for my ($i, $value) ($alpha->{beta}->[0]->%[ keys $alpha->{beta}->[0]->@* ]) { 38 | say "$i == $value"; 39 | } 40 | ``` 41 | 42 | This is pretty bad. 43 | 44 | With `indexed`, we write this: 45 | 46 | ```perl 47 | for my ($i, $value) (indexed $alpha->{beta}->[0]->@*) { 48 | say "$i == $value"; 49 | } 50 | ``` 51 | 52 | This is probably about as simple as this can get without some significant new 53 | addition to the language. 54 | 55 | ## Specification 56 | 57 | indexed LIST 58 | 59 | `indexed` takes a list of arguments. 60 | 61 | In scalar context, `indexed` evalutes to the number of entries in its argument, 62 | just like `keys` or `values`. This is useless, and issues a warning in the new 63 | "scalar" category: 64 | 65 | Useless use of indexed in scalar context 66 | 67 | In void context, the `Useless use of %s in void context` warning is issued. 68 | 69 | In list context, `indexed LIST` evalutes to a list twice the size of the list, 70 | meshing the values with a list of integers starting from zero. All values are 71 | copies, unlike `values ARRAY`. (If your LIST was actually an array, you can 72 | use the index to modify the array that way!) 73 | 74 | ## Backwards Compatibility 75 | 76 | There should be no significant backwards compatibility concerns. `indexed` 77 | will be imported only when requested. Static analysis tools may need to be 78 | updated. 79 | 80 | A polyfill for indexed can be provided for older perls, but may not be as 81 | optimizable. 82 | 83 | ## Security Implications 84 | 85 | Nothing specific predicted. 86 | 87 | ## Examples 88 | 89 | (See the examples under **Rationale**.) 90 | 91 | I expect that docs for `keys` and `values` will be updated to reference 92 | `indexed` as well, and we'll add a note about it to the documentation on `for` 93 | and possibly pair slices. 94 | 95 | When n-at-a-time foreach is no longer experimental, we should refer to the 96 | combination of `for my (...) (...)` with `indexed` as forming an alternative to 97 | `each` in the documentation for `each`. 98 | 99 | ## Prototype Implementation 100 | 101 | None. 102 | 103 | ## Future Scope 104 | 105 | I believe this will be complete as is. 106 | 107 | ## Rejected Ideas 108 | 109 | This proposal replaces one for `kv` which could be called on hash or array 110 | literals to act like a combination of `keys` and `values`. 111 | 112 | That proposal replaced one for a slice syntax that evaluated to a slice that 113 | omitted nothing. 114 | 115 | ## Open Issues 116 | 117 | None? 118 | 119 | ## Copyright 120 | 121 | Copyright (C) 2021, Ricardo Signes. 122 | 123 | This document and code and documentation within it may be used, redistributed 124 | and/or modified under the same terms as Perl itself. 125 | -------------------------------------------------------------------------------- /ppcs/ppc0017-original-build-type.md: -------------------------------------------------------------------------------- 1 | # built-in functions for checking the original type of a scalar 2 | 3 | ## Preamble 4 | 5 | Author: Graham Knop 6 | Sponsor: 7 | ID: 0017 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | This PPC proposes adding new builtin functions created_as_string and 13 | created_as_number for checking the original form of non-scalar references. 14 | Specifically, checking if a value was created as a string or a number. This 15 | is primarily meant for use by serializers. 16 | 17 | ## Motivation 18 | 19 | Perl itself is operator typed and is not generally meant to distinguish 20 | between values like 1 and "1". However, many external systems do care about 21 | these types, and many times these values need to be transferred through perl 22 | without anything like a schema to enforce these types. This makes it important 23 | for things like serializers to be able to track the "origin" type of a value. 24 | 25 | Classically, perl scalars have no set type. They may be created as strings or 26 | numbers, but based on their use they will internally store alternate forms of 27 | the value. If these alternate forms are considered "accurate", the 28 | corresponding flag will be set. This makes it impossible to know for certain 29 | how some values were created, as they will have both string and number forms 30 | that are flagged as "accurate". 31 | 32 | In perl v5.36, this has changed. The POK flag is now only set for values that 33 | are created as strings. Values that are created as strings may gain IOK or NOK 34 | flags based on their use, but since numbers will never gain the POK flag, this 35 | can still be used to detect numbers. 36 | 37 | Exposing this information via builtin functions will allow pure perl code to 38 | accurately serialize values. 39 | 40 | ## Rationale 41 | 42 | While the flags on scalars can be checked via the B module, using it requires 43 | an understanding of perl internals, and it is slow and uses a fairly large 44 | amount of memory. Being able to interpret the flags to know string vs number 45 | is also not entirely obvious. 46 | 47 | The functions are intentionally using longer less convenient names, with the 48 | hope that this will discourage their use for type checks within perl. Inside 49 | perl, 1 and "1" are meant to be treated equivalently. It is only when 50 | transferring data outside perl that these are meant to be used. 51 | 52 | `created_as` is meant to only tell how a value was created, without implying 53 | anything further about what the value "is". `created` in this case means any 54 | modification of the value, as any modification of a value requires creating 55 | the new, modified value. 56 | 57 | ## Specification 58 | 59 | created_as_string VALUE 60 | 61 | `created_as_string` takes a single scalar argument. 62 | 63 | It returns a boolean value representing if the value was originally created as 64 | a string. It will return false for references, numbers, booleans, and undef. 65 | Internally, this would be `SvPOK(sv) && !SvIsBOOL(sv)`. 66 | 67 | created_as_number VALUE 68 | 69 | `created_as_number` takes a single scalar argument. 70 | 71 | It returns a boolean value representing if the value was originally created as 72 | a number. It will return false for references, strings, booleans, and undef. 73 | Internally, this would be `SvNIOK(sv) && !SvPOK(sv) && !SvIsBOOL(sv)`. 74 | 75 | ## Backwards Compatibility 76 | 77 | There should be no significant backwards compatibility concerns. 78 | `created_as_string` and `created_as_number` will be imported only when 79 | requested. Static analysis tools may need to be updated. 80 | 81 | As these functions rely on the changes to how SV flags get set, it isn't 82 | possible to accurately implement these on older perls. Existing serializers 83 | use heuristics that will usually give accurate answers. One of these heuristics 84 | could be used to provide a compatible polyfill. 85 | 86 | ## Security Implications 87 | 88 | Nothing specific predicted. 89 | 90 | ## Examples 91 | ```perl 92 | use builtin qw(created_as_number created_as_string); 93 | 94 | my $value1 = "1"; 95 | my $string1 = created_as_string $value1; # true 96 | my $number1 = created_as_number $value1; # false 97 | 98 | my $value2 = 1; 99 | my $string2 = created_as_string $value2; # false 100 | my $number2 = created_as_number $value2; # true 101 | 102 | my $value3 = "1"; 103 | my $used_as_number3 = 0+$value3; 104 | my $string3 = created_as_string $value3; # true 105 | my $number3 = created_as_number $value3; # false 106 | 107 | my $value4 = 1; 108 | my $used_as_string4 = "$value4"; 109 | my $string4 = created_as_string $value4; # false 110 | my $number4 = created_as_number $value4; # true 111 | ``` 112 | 113 | ## Prototype Implementation 114 | 115 | None. 116 | 117 | ## Future Scope 118 | 119 | It is possible that in the future, the NOK or IOK flags will only be set for 120 | values created as numbers. That would mean the implementation of these 121 | functions may change, but should not impact the results returned. 122 | 123 | Additionally, if the NOK and IOK flags were only set for values created as 124 | integers or created as floats, it may be possible to provide additional 125 | functions `created_as_integer` and `created_as_float` to distinguish these 126 | types. Some serializers care about this distinction as well. 127 | 128 | ## Rejected Ideas 129 | 130 | Various alternative names for these functions have been proposed. 131 | 132 | `is_string` and `is_number` were the initial discussed functions, but these 133 | names imply stronger typing than perl is really meant to have. They would more 134 | strongly encourage using these functions for type checking within perl, rather 135 | than the usually more appropriate `looks_like_number`. 136 | 137 | `was_originally_string` and `was_originally_number` were also proposed, but 138 | `originally` carries the implication that the values may no longer be those 139 | things. The use of an adverb also feels wrong stylistically. 140 | 141 | ## Open Issues 142 | 143 | None? 144 | 145 | ## Copyright 146 | 147 | Copyright (C) 2022, Graham Knop. 148 | 149 | This document and code and documentation within it may be used, redistributed 150 | and/or modified under the same terms as Perl itself. 151 | -------------------------------------------------------------------------------- /ppcs/ppc0018-module-true.md: -------------------------------------------------------------------------------- 1 | # No Longer Require a True Value at the End of a Module 2 | 3 | ## Preamble 4 | 5 | Author: Curtis "Ovid" Poe 6 | Sponsor: 7 | ID: 0018 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | This PPC proposes a feature which, when used, eliminates the need to end a Perl 13 | module with the conventional "1" or other true value. 14 | 15 | ## Motivation 16 | 17 | Eliminate the need for a true value at the end of a Perl file. 18 | 19 | ## Rationale 20 | 21 | There's no need to have a true value be hard-coded in our files that we 22 | `use`. Further, newer programmers can get confused because sometimes code 23 | _doesn't_ end with a true value but nonetheless compiles just fine because 24 | _something_ in the code returned a true value and the code compiles as a 25 | side-effect. 26 | 27 | ## Specification 28 | 29 | First, a new `feature` is added: 30 | 31 | ```perl 32 | use feature 'module_true'; 33 | ``` 34 | 35 | Then, *whenever* a module is loaded with `require` (or an equivalent, like 36 | `use`), the "croak if false" test is skipped if the `module_true` feature was 37 | in effect at the last statement executed in the required module. 38 | 39 | ## Backwards Compatibility 40 | 41 | There are no compatibility concerns I'm aware of because we're only suggesting 42 | changing behaviour in the presence of a newly-added feature that is not 43 | present in any existing code. 44 | 45 | ## Security Implications 46 | 47 | None expected. 48 | 49 | ## Examples 50 | 51 | Imagine this module: 52 | 53 | ```perl 54 | package Demo1; 55 | use feature 'module_true'; 56 | 57 | sub import { 58 | warn "You imported a module!\n"; 59 | } 60 | ``` 61 | 62 | When loaded by `require` or `use` anywhere in perl, this would import 63 | successfully, despite the lack of a true value at the end. 64 | 65 | This module shows an (almost certainly never useful) way to croak anyway: 66 | 67 | ```perl 68 | package Demo2; 69 | use feature 'module_true'; 70 | 71 | return 1 if $main::test_1; 72 | return 0 if $main::test_2; 73 | 74 | { 75 | no feature 'module_true'; 76 | return 0 if $main::test_3; 77 | } 78 | ``` 79 | 80 | In this example, the only case in which requiring Demo2 would fail is if 81 | `$main::test_3` was true. The previous `return 0 if $main::test_2` would still 82 | be within the scope of the `module_true` feature, so the return value would be 83 | ignored. When `0` is returned outside the effect of `module_true`, though, the 84 | old behavior of testing the return value is back in effect. 85 | 86 | ## Prototype Implementation 87 | 88 | There is a prototype implementation at [true](https://metacpan.org/pod/true). 89 | 90 | ## Future Scope 91 | 92 | Due to this being a named feature, this can eventually be the default behavior 93 | when `use v5.XX;` is used. 94 | 95 | ## Rejected Ideas 96 | 97 | It's been discussed that we should return the package name instead. This 98 | supports: 99 | 100 | ```perl 101 | my $obj = (require Some::Object::Class)->new; 102 | ``` 103 | 104 | However, per haarg: 105 | 106 | > Changing the return value of require seems like a separate issue from what 107 | > this PPC wants to address. 108 | > 109 | > If you wanted require to always return the same value, and for that value to 110 | > come from the file, you need a new location to store these values. This 111 | > would probably mean a new superglobal to go along with %INC. And it would 112 | > usually be useless because most modules return 1. I don't think this is a 113 | > very good idea. 114 | > 115 | > If you wanted require to always return the package name, it's a separate 116 | > issue from this PPC because that means ignoring the return value from the 117 | > file. It also presents a problem because require doesn't actually take 118 | > package names. It takes file path fragments. Foo::Bar is translated to 119 | > "Foo/Bar.pm" at parse time. This would then need to be converted back to a 120 | > package name, or do something else. I don't think this is a good idea 121 | > either. 122 | > 123 | > Instead, it's probably best addressed with a builtin::load or similar 124 | > routine that accepts a package as a string. This has been discussed in the 125 | > past, and solves other problems. Module::Runtime has a use_module function 126 | > that behaves like this, returning the package name. 127 | 128 | Thus, we prefer simply returning `true` (or `1`). 129 | 130 | ## Copyright 131 | 132 | Copyright (C) 2022, Curtis "Ovid" Poe 133 | 134 | This document and code and documentation within it may be used, redistributed 135 | and/or modified under the same terms as Perl itself. 136 | -------------------------------------------------------------------------------- /ppcs/ppc0019-qt-string.md: -------------------------------------------------------------------------------- 1 | # String Literals with Expression Interpolation 2 | 3 | ## Preamble 4 | 5 | Author: Ricardo Signes 6 | Sponsor: RJBS 7 | ID: 0019 8 | Status: Draft 9 | 10 | ## Abstract 11 | 12 | This document proposes a new quote-like operator for string literals. This 13 | operator, `qt`, is meant to serve as an alternative to `qq`, with simpler rules 14 | for understanding what is interpolated and how. It takes its design from 15 | JavaScript's template literals and Ruby's string interpolation. 16 | 17 | Instead of allowing a subset of variable expressions directly within a string, 18 | qt allows *any* expression to be interpolated when delimited within the string 19 | literal. 20 | 21 | ## Motivation 22 | 23 | Existing interpolating strings are extremely convenient, but have shortcomings 24 | built into their design. Examples: 25 | 26 | ```perl 27 | my $what = "balloons"; 28 | my $colors = [ qw( red white blue ) ]; 29 | 30 | my $object = Party->new({ type => "birthday", bring => $what }); 31 | 32 | $s = "$problems"; # "" . $problems 33 | $s = "$colors->[0]"; # "" . $colors->[0] 34 | 35 | $s = "@$colors"; # join($", @$colors) 36 | $s = "$colors->@*"; # If postderef_qq on: join($", $colors->@*) 37 | # Otherwise : "" . $colors . "->@*" 38 | 39 | $s = "Bring $object->{bring}"; # "Bring " . $object->{bring} 40 | $s = "Bring $object->bring"; # "Bring " . $object . "->bring" 41 | ``` 42 | 43 | As we look into adding new forms of dereference (for example the proposed `?->` 44 | operator), or at figuring out how to make `postderef_qq` a default behavior, we 45 | are likely to keep hitting confusing problems with the behavior of `qq`. We 46 | should add something that designs away all these problems now and for the 47 | future. 48 | 49 | ## Rationale 50 | 51 | The proposed `qt` operator only looks for one special token in a string 52 | literal: `{`. Source content until the matching `}` is treated as a scalar 53 | expression to be interpolated into the string in place of the `{...}` 54 | construct. This is arbitrarily extensible for other existing expressions and 55 | for new ones that may be added. 56 | 57 | To include a verbatim `{` in the string value, escape it: `\{` 58 | 59 | ## Specification 60 | 61 | This: 62 | 63 | ``` 64 | qt{A { TEXT } B}; 65 | ``` 66 | 67 | will be equivalent to 68 | 69 | ``` 70 | qq{A ${ \scalar(TEXT) } B}; 71 | ``` 72 | 73 | The `qt` operator recognises the same escape sequences as `qq` that represent 74 | other literal characters (such as `\n` for newline or `\x00` for a NUL byte). 75 | 76 | ```perl 77 | print qt{This message ends in a newline\n}; 78 | 79 | print qt{As does this message\x0d\x0a}; 80 | 81 | print qt{It's Christmas! \N{SNOWMAN} }; 82 | ``` 83 | 84 | The `$` and `@` symbols are *not* considered special and will act literally, 85 | as they behave inside a `q` string. 86 | 87 | ```perl 88 | print qt{This apple will cost you $1.\n}; 89 | # will print a literal '$' symbol followed by the digit 1, it does not 90 | # interpolate the contents of the first regexp capture buffer 91 | ``` 92 | 93 | Likewise, the escape sequences `\L`, `\l`, `\U`, `\u`, `\Q` and `\E`, which in 94 | `qq` strings control the way that variables are interpolated, are also not 95 | considered special here. It would be helpful to users to at least issue a 96 | warning in this case, to remind them of this fact. 97 | 98 | ``` 99 | $ perl -E 'say qt{This does not uppercase the variable \U{ $var }};' 100 | Unrecognised escape \U passed through at FILE line LINE. 101 | ``` 102 | 103 | To provide a heredoc version of qt, this: 104 | 105 | ``` 106 | <; 156 | 157 | # Interpolation of method calls 158 | qt"Greetings, {$user->title} {$user->name}"; 159 | 160 | # Interpolation of various expressions 161 | qt{It has been {$since{n}} {$since{units}} since your last login}; 162 | 163 | qt{...a game of {join q{$"}, $favorites->{game}->name_words->@*}}; 164 | ``` 165 | 166 | ## Prototype Implementation 167 | 168 | Quote::Code, mentioned above, provides something very similar. 169 | 170 | ## Future Scope 171 | 172 | `qt` is "qq but different". If successful, it might suggest the usefulness of 173 | qt-like forms of qr, m, s, and other string-interpolating contexts. This also 174 | suggests that another possible design for qt would be a pragma to change the 175 | behavior of qq *and other interpolating quote-like operators*. This is more 176 | complex for the reader (because any given piece of code must be considered in 177 | terms of the enabled features), but eliminates the proliferation of QLOPs. 178 | 179 | ## Rejected Ideas 180 | 181 | This proposal uses `{...}` instead of `${...}` because `${...}` expects a 182 | *reference* inside in all other contexts, which is not the case here. 183 | 184 | This proposal uses `{...}` (like JavaScript) instead of `#{...}` (like Ruby) 185 | because the author didn't think the extra character was likely valuable enough 186 | for disambiguation between literal `{` and start of interpolated expression. 187 | 188 | This proposal does not offer a means to interpolate a list without (say) 189 | `join`. It would require more syntax and leave the user falling back on 190 | remembering that multiple arguments to `print` are joined with one thing (`$,`, 191 | usually an empty string) but multiple list elements interpolated into a string 192 | are joined with another (`$"`, usually a space). 193 | 194 | Quote::Code has `qc_to` for heredocs instead of a qc-quoted heredoc terminator. 195 | Matching "terminator quoting determines heredoc interpolation" seemed more 196 | "keep similar things similar". 197 | 198 | ## Open Issues 199 | 200 | What, if anything, do we do now about interpolating into regex? 201 | 202 | ## Copyright 203 | 204 | Copyright (C) 2022, Ricardo Signes. 205 | 206 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 207 | -------------------------------------------------------------------------------- /ppcs/ppc0020-lexical-export.md: -------------------------------------------------------------------------------- 1 | # A builtin for Lexical Export 2 | 3 | ## Preamble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0020 8 | Status: Implemented 9 | 10 | ## Abstract 11 | 12 | Add a new function to the `builtin` package to perform lexical export into the scope currently being compiled. 13 | 14 | ## Motivation 15 | 16 | "Traditional" Perl modules of the era around Perl 5.8 have always used symbolic aliasing into the caller's package to implement their export mechanism. This was the only mechanism available to them at the time, and served well in the era of procedural modules with functions in them. As more code became object-oriented, using packages to implement classes, the problems of making these exports visible as named symbols in the class's namespace became apparent, because each imported function would be visible as a named method on the class, whether it wanted that or not. 17 | 18 | A new feature of the Perl 5.18 release was the ability to create lexically-named subroutines, whose names are accessible within some lexical scope but not visible in the package namespace, and thus not visible to callers from outside the code, even via method reflection. Since then, the ability has not been further expanded on by Perl core. While a few CPAN modules have tried to implement the ability, the idea does not seem to have caught on in the majority of cases. 19 | 20 | The `builtin` module added in Perl 5.36 was the first core module to export its named functions lexically into the calling scope, rather than symbolically into the caller's package. It aims to set a new standard for better-behaved module exporting, but is currently handled by its own internal implementation which is not yet exposed in a way that other modules can make use of. 21 | 22 | ## Rationale 23 | 24 | By providing a mechanism by which other modules written in pure perl code can take advantage of lexical export themselves, we hope that more will begin to take advantage of this ability. This will lead to modules that behave more politely in the face of the growing collection of modules written in object-oriented style. In addition, as addressed by the "Future Scope" section, we hope this leads on to the creation of better Exporter-like modules or other core perl abilities to take advantage of lexical export. 25 | 26 | ## Specification 27 | 28 | A new function in the `builtin` package, called `export_lexically`. This function takes an even-sized name/value list of pairs. It does not return anything. 29 | 30 | ```perl 31 | builtin::export_lexically $name1, $value1, $name2, $value2, ...; 32 | ``` 33 | 34 | Each name/value pair acts independently. The name gives the new name to be created in the scope being compiled, and the value is a reference to the item the name should have. 35 | 36 | Four kinds of items are supported: 37 | 38 | + If the value is a `CODE` reference, a new named subroutine will be created in the scope. The corresponding name argument may optionally begin with a `&` sigil. 39 | 40 | + If the value is a `SCALAR` reference, a new scalar variable will be created in the scope. The corresponding name argument must begin with a `$` sigil. 41 | 42 | + If the value is an `ARRAY` reference, a new array variable will be created in the scope. The corresponding name argument must begin with a `@` sigil. 43 | 44 | + If the value is a `HASH` reference, a new hash variable will be created in the scope. The corresponding name argument must begin with a `%` sigil. 45 | 46 | If no sigil is present on the name argument, it is presumed to be of the first kind, and the value must be a `CODE` reference. 47 | 48 | As the only effect this function has is to add new things to the compiletime lexical scope, it shall be an error to call this function during regular runtime, when no code is currently being compiled. It shall also be an error to pass non-reference values, references to unsupported items (e.g. globs), or references whose type does not match the sigil of its name. 49 | 50 | ## Backwards Compatibility 51 | 52 | As this specification adds one new optionally-requested builtin function name there are not expected to be any backward compatiblity concerns. Additionally, the concept of lexical exports and lexically-named subroutines has existed in Perl since version 5.18, so these are not new concepts. 53 | 54 | ## Security Implications 55 | 56 | None are expected. 57 | 58 | ## Examples 59 | 60 | ### A simple direct application: 61 | 62 | ```perl 63 | { 64 | BEGIN { 65 | builtin::export_lexically ten => sub { 10 }; 66 | } 67 | 68 | say "Ten plus ten is ", ten() + ten(); 69 | } 70 | 71 | # ten() is no longer visible here 72 | ``` 73 | 74 | ### A tiny module exporting lexically 75 | 76 | ```perl 77 | package Example::Module; 78 | 79 | sub import 80 | { 81 | shift; 82 | 83 | my %syms; 84 | $syms{$_} = __PACKAGE__->can( $_ ) // die "No such function '$_'" 85 | for @_; 86 | 87 | builtin::export_lexically %syms; 88 | } 89 | 90 | sub foo { ... } 91 | sub bar { ... } 92 | 93 | 1; 94 | ``` 95 | 96 | ## Prototype Implementation 97 | 98 | An initial implementation of this specification has been written at https://github.com/leonerd/perl5/tree/lexically-export, and raised as a Pull Request against core perl5 at https://github.com/Perl/perl5/pull/19895. 99 | 100 | ## Future Scope 101 | 102 | Once a basic mechanism for lexical export exists in the language, it becomes tempting to look at the various Exporter-like modules, either bundled with perl (e.g. [`Exporter`](https://metacpan.org/pod/Exporter) itself) or on CPAN. Various discussions in the past have raised the idea of having those modules perform lexical export of the requested names. 103 | 104 | Another possible idea becomes the thought of adding an `:export` attribute for functions and variables, bundling most of the export ability directly into the core language and avoiding most of the need to invoke an external module to do it. 105 | 106 | Both of these ideas are out of scope for this PPC, but would be made significantly easier by its completion. 107 | 108 | ## Rejected Ideas 109 | 110 | ## Open Issues 111 | 112 | ## Copyright 113 | 114 | Copyright (C) 2022, Paul Evans. 115 | 116 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 117 | -------------------------------------------------------------------------------- /ppcs/ppc0023-map-grep-with-topic.md: -------------------------------------------------------------------------------- 1 | # Map with different topic variable 2 | 3 | ## Preamble 4 | 5 | Author: Graham Knop 6 | ID: 0023 7 | Status: Exploratory 8 | 9 | ## Abstract 10 | 11 | Allow `map` and `grep` to be given a different variable to be used as the topic 12 | variable, rather than using `$_`. Also allow multiple variables to be given to 13 | do n-at-a-time iteration. 14 | 15 | ## Motivation 16 | 17 | Traditionally, `map` and `grep` loop over a list, aliasing `$_` to each list 18 | entry. While this can be convenient in many cases, it gets awkward when nested 19 | maps are needed. You need to manually save the `$_` value in another variable. 20 | 21 | ```perl 22 | my %hash = ( 23 | foo => [ 1, 2, 3, 4 ], 24 | bar => [ 5, 6, 7, 8 ], 25 | ); 26 | 27 | my @out = map { 28 | my $key = $_; 29 | map { "$key: $_" } $hash{$key}->@*; 30 | } keys %hash; 31 | ``` 32 | 33 | Using `$_` can also be dangerous if you need to call code not under your 34 | direct control, due to it being the implicit target of operations like 35 | `readline`. 36 | 37 | It would be more convenient if you could use a different variable for the 38 | topic, similar to what `for` allows. 39 | 40 | ```perl 41 | my @out = map my $key { 42 | map my $i { "$key: $i" } $hash{$key}->@*; 43 | } keys %hash; 44 | ``` 45 | 46 | A natural extension of this syntax would be to allow n-at-a-time iteration, as 47 | `for` can do on perl 5.36+. 48 | 49 | ```perl 50 | my @out = map my ($key, $val) { 51 | map my $i { "$key: $i" } $val->@*; 52 | } %hash; 53 | ``` 54 | 55 | All of this would apply to `grep` as well. 56 | 57 | As the syntax proposed is currently invalid, a feature should not be needed to 58 | enable it. 59 | 60 | ## Rationale 61 | 62 | The syntax chosen is meant to follow from the syntax of `for`, treating `map` 63 | as "`for` as an expression". The chosen syntax does not create any 64 | ambiguities, and naturally extends to follow the syntax used by `for` for 65 | n-at-a-time iteration. 66 | 67 | ## Specification 68 | 69 | ### `map my VAR BLOCK LIST` 70 | 71 | This will evaluate `BLOCK` for each element of `LIST`, aliasing `VAR` to each 72 | element. Its behavior will otherwise match `map BLOCK LIST`. It is only 73 | possible to use `my` variables for this. 74 | 75 | ### `grep my VAR BLOCK LIST` 76 | 77 | This will evaluate `BLOCK` for each element of `LIST`, aliasing `VAR` to each 78 | element. Its behavior will otherwise match `grep BLOCK LIST`. It is only 79 | possible to use `my` variables for this. 80 | 81 | ### `map my (VAR, VAR) BLOCK LIST` 82 | 83 | This will evaluate `BLOCK` for each set of two elements in `LIST`, aliasing 84 | the `VAR` to the first of each set, and the second `VAR` to the second. More 85 | than two variables can be used to iterate over sets of three or more items. 86 | 87 | On the last iteration, there may not be enough elements remaining to fill 88 | every `VAR` slot. In this case, a warning will be issued, and the extra `VAR` 89 | slots will be filled with `undef`. 90 | 91 | In scalar context, the return value will be the number of elements generated. 92 | 93 | ### `grep my (VAR, VAR) BLOCK LIST` 94 | 95 | This will evaluate `BLOCK` for each set of two elements in `LIST`, aliasing 96 | the `VAR` to the first of each set, and the second `VAR` to the second. More 97 | than two variables can be used to iterate over sets of three or more items. 98 | 99 | On the last iteration, there may not be enough elements remaining to fill 100 | every `VAR` slot. In this case, a warning will be issued, and the extra `VAR` 101 | slots will be filled with `undef`. The extra `undef` values used will also be 102 | included in the returned list. 103 | 104 | In scalar context, this will return number of elements that would have been 105 | generated. This means it will always be a multiple of the count of `VAR` 106 | slots. 107 | 108 | ## Backwards Compatibility 109 | 110 | As the syntax chosen is currently invalid, it should not present any backwards 111 | compatibility concerns. 112 | 113 | While the parsing of `map` and `grep` is complex and has ambiguities, the 114 | additions proposed are bounded and simple to parse, so they do not introduce 115 | any new ambiguities. 116 | 117 | ## Security Implications 118 | 119 | None foreseen. 120 | 121 | ## Examples 122 | 123 | ```perl 124 | my @my_array = (11 .. 14); 125 | 126 | my @trad_map = map { $_ + 1 } @my_array; 127 | # ( 12, 13, 14, 15 ) 128 | 129 | my @new_map = map my $f { $f + 1 } @my_array; 130 | # ( 12, 13, 14, 15 ) 131 | 132 | my @pair_map = map my ($f, $g) { $f + $g } @my_array; 133 | # ( 23, 27 ) 134 | 135 | my @new_grep = grep my $f { $f > 12 } @my_array; 136 | # ( 13, 14 ) 137 | 138 | my %my_hash = ( 139 | first_key => 11, 140 | second_key => 14, 141 | ); 142 | 143 | my %pair_grep = grep my ($f, $g) { $f =~ /sec/ } %my_hash; 144 | # ( second_key => 14 ) 145 | ``` 146 | 147 | ## Prototype Implementation 148 | 149 | - List::Util includes the functions `pairmap` and `pairgrep` which can do 150 | 2-at-a-time iteration. These use the "magic" `$a` and `$b` variables. 151 | 152 | ## Future Scope 153 | 154 | - Doing n-at-a-time iteration is useful, but authors may not have a use for 155 | all of the variables, especially with `grep`. Could the extra variables be 156 | eliminated? Maybe `grep my ($key, undef) { ... } ...` could be supported? 157 | 158 | - `map`, `grep`, and even `for` could possibly be extended to allow 159 | refaliasing syntax for their variables. 160 | 161 | ```perl 162 | my %hash = ( 163 | key1 => [ 1, 2, 3 ], 164 | key2 => [ 4, 5, 6 ], 165 | ); 166 | my %out = map my ($key, \@values) { ... } %hash; 167 | ``` 168 | 169 | - Subs with a `(&@)` prototype are, in part, meant to allow mimicking the 170 | syntax of `map` and `grep`. An example of this being `List::Util::any`: 171 | 172 | ```perl 173 | use List::Util qw(any); 174 | 175 | my $found = any { /ab/ } qw( 1234 abcd ); 176 | ``` 177 | 178 | It would be desirable to be able to to write a perl function that accepted 179 | the same syntax this this PPC proposes. 180 | 181 | ## Rejected Ideas 182 | 183 | - `my` is required because the behavior of `for` when used with non-`my` or 184 | predeclared variables is confusing and hard to explain. The newer syntax 185 | `for my ($var1, $var2) { ... }` also requires `my`, both for that reason 186 | as well as implementation details. Making the `my` implicit also 187 | introduces a difference from `for` making it harder to document and 188 | explain. 189 | 190 | ## Open Issues 191 | 192 | Nothing at this time. 193 | 194 | ## References 195 | 196 | - Pre-PPC discussion: https://www.nntp.perl.org/group/perl.perl5.porters/2022/11/msg265104.html 197 | 198 | ## Copyright 199 | 200 | Copyright (C) 2022, Graham Knop 201 | 202 | This document and code and documentation within it may be used, redistributed 203 | and/or modified under the same terms as Perl itself. 204 | -------------------------------------------------------------------------------- /ppcs/ppc0024-signature-named-parameters.md: -------------------------------------------------------------------------------- 1 | # Named Parameters in Signatures 2 | 3 | ## Preamble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0024 8 | Status: Exploratory 9 | 10 | ## Abstract 11 | 12 | Adds the ability for subroutine signatures to process named arguments in the form of name-value pairs passed by the caller, in a fashion familiar to existing uses of assignment into a hash. 13 | 14 | ## Motivation 15 | 16 | Perl 5.20 added "subroutine signatures", native syntax for more succinctly expressing the common patterns of processing arguments inbound into a subroutine by unpacking the `@_` array. Rather than writing common styles of code directly, a more compact notation, in a style immediately familiar to users of many other languages, allows Perl to create the required behaviour from that specification. 17 | 18 | ```perl 19 | sub f ($x, $y, $z = 123) { ... } 20 | 21 | # instead of 22 | sub f { 23 | die "Too many arguments" if @_ > 3; 24 | die "Too few arguments" if @_ < 2; 25 | my ($x, $y, $z) = @_; 26 | $z = 123 if @_ < 3; 27 | ... 28 | } 29 | ``` 30 | 31 | One common form of argument processing involves passing an even-sized list of key/value pairs and assigning that into a hash within the subroutine, so that specifically named parameters can be extracted from it. There is currently no support from subroutine signature syntax to assist authors in providing such behaviours. 32 | 33 | ## Rationale 34 | 35 | (explain why the (following) proposed solution will solve it) 36 | 37 | ## Specification 38 | 39 | A new kind of element may be present in a subroutine signature, which consumes a named argument from the caller. These elements are written with a leading colon prefix (`:$name`), indicating that it is named rather than positional. The name of each parameter is implied by the name of the lexical into which it is assigned, minus the leading `$` sigil. 40 | 41 | Each element provides a new lexical variable that is visible during the body of the function, in the same manner as positional ones. 42 | 43 | The value of a named parameter is taken from the argument values passed by the caller, in a manner familiar to existing uses of hash assignment. The caller should pass an even-sized name-value pair list. The values corresponding to names of parameters will be assigned into the variables. The order in which the values are passed by the caller is not significant. 44 | 45 | Since it is a relatively common pattern in callsites in existing code to rely on the semantics of assignment of name-value pair lists into hashes, the beahviour on encountering duplicate key names needs to be preserved. This is that duplicated key names do not raise an error or a warning, and simply accept the last value associated with that name. This allows callers to collect values from multiple sources with different orders of priority to override them; for example using a hash of values combined with individual elements: 46 | 47 | ```perl 48 | sub func ( :$abc, :$xyz, ... ) { ... } 49 | 50 | func( 51 | abc => 123, 52 | %args, 53 | xyz => 789, 54 | ); 55 | ``` 56 | 57 | In this example, the given `abc` value will take effect unless overridden by a later value in `%args`, but the given `xyz` value will replace an earlier one given in `%args`. Neither situation will result in a warning or error. 58 | 59 | Furthemore, all of the new behaviour is performed within the body of the invoked subroutine entirely by inspecting the values of the arguments that were passed. The subroutine is not made aware of how those values came to be passed in - whether from literal name-value syntax, a hash or array variable expansion, or any other expression yielding such a list of argument name and value pairs. 60 | 61 | ```perl 62 | sub make_colour ( :$red, :$green, :$blue ) { ... } 63 | 64 | make_colour( red => 1.0, blue => 0.5, green => 0.2 ); 65 | # The body of the function will be invoked with 66 | # $red = 1.0 67 | # $green = 0.2 68 | # $blue = 0.5 69 | ``` 70 | 71 | As with positional parameters, a named parameter without a defaulting expression is mandatory, and an error will be raised as an exception if the caller fails to pass a corresponding value. A defaulting expression may be specified using any of the operators available to positional parameters - `=`, `//=` or `||=`. 72 | 73 | ```perl 74 | sub make_colour ( :$red = 0, :$green = 0, :$blue = 0 ) { ... } 75 | 76 | make_colour( red => 1.0, blue => 0.5 ); 77 | # The body of the function will be invoked with 78 | # $red = 1.0 79 | # $green = 0 80 | # $blue = 0.5 81 | ``` 82 | 83 | Since defaulting expressions are full expressions and not necessarily simple constant values, the time at which they are evaluated is significant. Much like with positional parameters, each is evaluated in order that it is written in source, left to right, and each can make use of values assigned by earlier expressions. This means they are evaluated in the order that is written in the function's declaration, which may not match the order that values were passed from the caller in the arguments list. 84 | 85 | A subroutine is permitted to use a combination of *mandatory* positional and named parameters in its definition, provided that all named parameters appear after all the positional ones. Any named parameters may be optional; there are no ordering constraints here. 86 | 87 | If a subroutine uses named parameters then it may optionally use a slurpy hash argument as its final element. In this case, the hash will receive all *other* name-value pairs passed by the caller, apart from those consumed by named parameters. If the subroutine does not use a slurpy argument, then it will be an error raised as an exception for there to be any remaining name-value pairs after processing. 88 | 89 | While all of the above has been specified in terms of subroutines, every point should also apply equally to the methods provided by the `class` feature, after first processing the implied invocant `$self` parameter. 90 | 91 | ## Backwards Compatibility 92 | 93 | At the site of a subroutine's definition, this specification only uses new syntax in the form of the leading colon prefix on parameter names. Such syntax was formerly invalid in previous versions of perl. Thus there is no danger of previously-valid code being misinterpreted. 94 | 95 | All of the behaviour provided by this new syntax is compatible with and analogous to any existing code that could have been written prior, perhaps by direct assignment into a hash. There are no visible differences in the external interface to a subroutine using such syntax, and it remains callable in exactly the same way. Functions provided by modules could be upgraded to use the new syntax without any impact on existing code that invokes them. 96 | 97 | ## Security Implications 98 | 99 | There are no anticipated security concerns with expanding the way that subroutines process parameters in this fashion. 100 | 101 | ## Examples 102 | 103 | As the intention of this syntax addition is to make existing code practice more consise and simple to write, it would be illustrative to compare pairs of functions written in the newly proposed vs. the existing style. 104 | 105 | One example comes from [`IPC::MicroSocket::Server`](https://metacpan.org/pod/IPC::MicroSocket::Server). This is currently using the `Sublike::Extended` module (see "Prototype Implementation" below) to provide the syntax. The actual module also uses `Object::Pad` to provide the `method` keyword; this example is paraphrased to avoid that. This module requires a `path` named argument, and optionally takes a `listen` argument, defaulting its value to 5 if the caller did not provide a defined value. 106 | 107 | ```perl 108 | extended sub new_unix ( $class, :$path, :$listen //= 5 ) 109 | { 110 | ... 111 | } 112 | ``` 113 | 114 | This replaces the previous version of the code which handled arguments by the more traditional approach of assigning into a hash: 115 | 116 | ```perl 117 | sub new_unix ( $class, %args ) 118 | { 119 | my $path = $args{path}; 120 | my $listen = $args{listen} // 5; 121 | ... 122 | } 123 | ``` 124 | 125 | Already the new code is shorter and more consise. Additionally it contains error checking that complains about missing mandatory keys, or unrecognised keys, which the previous version of the code did not include. 126 | 127 | Another example, this time from [`Text::Treesitter::QueryCursor`](https://metacpan.org/pod/Text::Treesitter::QueryCursor). This method takes an optional argument named `multi`, tested for truth. If absent it should default to false. 128 | 129 | ```perl 130 | sub next_match_captures ( $self, :$multi = 0 ) 131 | { 132 | ... 133 | } 134 | ``` 135 | 136 | The previous version of this code did include complaints about unrecognised keys, and was rather longer because of it: 137 | 138 | ```perl 139 | sub next_match_captures ( $self, %options ) 140 | { 141 | my $multi = delete $options{multi}; 142 | keys %options and 143 | croak "Unrecognised options to ->next_captures: " . join( ", ", keys %options ); 144 | 145 | ... 146 | } 147 | ``` 148 | 149 | ## Prototype Implementation 150 | 151 | The [`XS::Parse::Sublike`](https://metacpan.org/pod/XS::Parse::Sublike) module contains parsing to allow third-party syntax modules to parse subroutine-like constructions, and includes a parser for named parameters already having this syntax. These are also made available to regular subroutines via the `extended` keyword provided by [`Sublike::Extended`](https://metacpan.org/pod/Sublike::Extended). 152 | 153 | Additionally, other existing CPAN modules already parse syntax in this, or a very similar format: 154 | 155 | * [`Function::Parameters`](https://metacpan.org/pod/Function::Parameters) 156 | 157 | * [`Kavorka`](https://metacpan.org/dist/Kavorka/view/lib/Kavorka/Manual/Signatures.pod) 158 | 159 | * [`Method::Signatures`](https://metacpan.org/pod/Method::Signatures) 160 | 161 | All of these use the leading-colon syntax in a signature declaration to provide named parameters in the same style as this proposal. It would appear we are in good company here. 162 | 163 | ## Future Scope 164 | 165 | * If the Metaprogramming API (PPC0022) gains introspection abilities to enquire about subroutine signature parameters, further consideration will need to be made in that API on how to represent the extra kinds of parameters added by this specification. 166 | 167 | ## Rejected Ideas 168 | 169 | * This specification intentionally makes no changes to the call-site of any subroutines using named parameters. 170 | 171 | ## Open Issues 172 | 173 | ### Parameter Names 174 | 175 | How to specify a named parameter whose name is anything other than the name of the lexical variable into which its value is assigned? This is more of note when considering that traditional argument handling techniques involving assignment into hashes can handle more styles of name that would be invalid for lexical variables - for example, names including hyphens. 176 | 177 | Perhaps a solution would be to permit attributes on parameter variables, then define a `:name` attribute for this purpose. 178 | 179 | ```perl 180 | sub display ( $message, :$no_colour :name(no-colour) ) { 181 | ... 182 | } 183 | ``` 184 | 185 | Further thoughts in that direction suggested that this could also support multiple names with aliases: 186 | 187 | ```perl 188 | sub display ( $message, :$no_colour :name(no-colour no-color) ) { 189 | ... 190 | } 191 | ``` 192 | 193 | The attribute syntax starting with a leading colon does visiually look quite similar to the named parameter syntax which also uses. This is a little unfortunate, but perhaps an inevitable consequence of the limited set of characters available from ASCII. As attributes are already well-established as leading with a colon, the only other option would be to pick a different character for named attributes; but this would be contrary to the established convention of the existing modules listed above. 194 | 195 | Perl does not currently support attributes being applied to parameters in signatures, named or otherwise, but this is the subject of a future PPC document I am currently drafting. I will add a reference here when it is published. 196 | 197 | ## Copyright 198 | 199 | Copyright (C) 2024, Paul Evans. 200 | 201 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 202 | -------------------------------------------------------------------------------- /ppcs/ppc0026-enhanced-regex-xx.md: -------------------------------------------------------------------------------- 1 | # RFC - enhanced regex /xx 2 | 3 | ## Preamble 4 | 5 | Author: Karl Williamson 6 | ID: 0026 7 | Status: Draft 8 | 9 | ## Abstract 10 | 11 | Let programmers improve the readability of regular expression patterns beyond 12 | what is possible now. 13 | 14 | ## Motivation 15 | 16 | Regular expression patterns were designed for concision rather than clarity. 17 | 18 | The /x regular expression pattern modifier was created to enable adding 19 | comments and white space to patterns to make them more readable. It suffers 20 | from not working for bracketed character classes, and silently compiling to 21 | something unintended when the programmer forgets to mark literal white space, 22 | and much worse, literal '#'. This last silently swallows the rest of the line 23 | that was supposed to be a part of the pattern. 24 | 25 | I eventually added /xx to at least allow tabs and blanks inside bracketed 26 | character classes. This allows a very minor improvement in their readability. 27 | I could not figure out a way to extend this to allow comments and multiple 28 | lines inside such a class without making it even more likely that the pattern 29 | would silently compile to something unintended. But now, I think this RFC 30 | fixes that. 31 | 32 | ## Specification 33 | 34 | I propose adding a new opt-in feature. Call it, for now, "feature 35 | `enhanced_re_xx`". Within its scope, the /xx modifier would change things so 36 | that inside a bracketed character class [...], any vertical space would be 37 | treated as a blank, essentially ignored. Any unescaped '#' would begin a 38 | comment that ends at the end of the line. 39 | 40 | This would change the existing /x behavior where the portion of the line after 41 | the '#' is parsed, looking for a potential pattern terminating delimiter. 42 | Under this feature to terminate a pattern, do so before any '#' on a line. 43 | If an unescaped terminating delimiter is found after a '#' on a line, a warning 44 | would be raised. 45 | 46 | And an unescaped '#' within a comment would raise a warning. So 47 | 48 | ``` 49 | $a[$i] =~ qr/ [ a-z # We need to match the lowercase alphabetics 50 | ! @ # . * # And certain punctuation 51 | 0-9 # And the digits (which can only occur in $a[0]) 52 | ] 53 | /xx; 54 | ``` 55 | 56 | would warn. 57 | 58 | It might be that an unescaped '#' that isn't of the form \s+#\s+ should 59 | warn to catch things like if the above example's second line were just 60 | 61 | ``` 62 | !@#.* 63 | ``` 64 | 65 | Also, any comments inside [...] would check for an unescaped ']' on the same 66 | line after a '#', and raise a warning if found. So, something like 67 | 68 | ``` 69 | $a[$i] =~ qr/ [ a-z # . * ] 70 | [ A-Z ] 71 | /xx; 72 | ``` 73 | 74 | would warn. Either escape the '#' or the ']' to suppress it, depending on what 75 | your intent was. 76 | 77 | I think these would catch essentially all unintended uses of '#' to mean 78 | not-a-comment, but to be taken literally. 79 | 80 | I can't think of anything to catch blanks/tabs being unintentionally ignored. 81 | 82 | I also propose that unescaped '#' and vertical space inside bracketed character 83 | classes under /xx be deprecated. /xx has been available only since 5.26; 84 | there's not a huge amount of code that uses it. After the deprecation cycle, 85 | the feature could become automatic, not opt-in, and /xx would have the new 86 | meaning. 87 | 88 | Note there is no change to plain /x. 89 | 90 | Copyright (C) 2022 Karl Williamson 91 | 92 | This document and code and documentation within it may be used, redistributed 93 | and/or modified under the same terms as Perl itself. 94 | 95 | -------------------------------------------------------------------------------- /ppcs/ppc0027-any-and-all.md: -------------------------------------------------------------------------------- 1 | # More list processing operators inspired by List::Util 2 | 3 | ## Preamble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0027 8 | Status: Exploratory 9 | 10 | ## Abstract 11 | 12 | Creates several new list-processing operators, similar to the existing `grep`, inspired by the same-named functions in modules like `List::Util` or `List::Keywords`. 13 | 14 | ## Motivation 15 | 16 | Most code of any appreciable size tends to make use of at least the `any` or `all` functions from `List::Util`. Due to limits of their implementation they are not as efficient to call as core's `grep` operator. The implementations provided by `List::Keywords` are more efficient, on the same level as core's `grep`, though being a non-core module it does not appear to be used anywhere near as much in practice. 17 | 18 | ## Rationale 19 | 20 | ## Specification 21 | 22 | New named features that, when enabled, activate syntax analogous to the existing `grep` operator, named `any` and `all`: 23 | 24 | ```perl 25 | any { BLOCK } LIST 26 | 27 | all { BLOCK } LIST 28 | ``` 29 | 30 | These operators are similar to `grep` in scalar context, though yield a simple boolean truth value relating to how many input values made the filter block yield true. `any` yields true when its block yields true for at least one of its input values, or false if they all yield false. `all` yields true only if every input value makes the block yield true, or false if at least one yields false. 31 | 32 | A consequence of these rules is what happens when given an empty list. A call to `any` with an empty list does not have any input values which made the block return true, so its result is false. Conversely, a call to `all` with an empty list does not have any input values which made the block return false, so its result is true. 33 | 34 | The key difference between these operators and `grep` is that these will short-circuit and stop evaluating the block once its result is determined. In particular, the first time `any` sees a true result, it knows its result so it can stop; only testing more input values while each yields false. Conversely, `all` will stop as soon as it sees a false result, knowing that to be its answer; it only continues while each value yields true from the block. This short-circuiting is a key reason to choose these over the `grep` operator. 35 | 36 | Additionally, because each operator returns a fixed boolean truth value, the caller does not have to take special precautions against a value that would appear false, which satisfies the filter code block. Such a value would cause the operator to return true, even if the matching value itself appears false. 37 | 38 | Like `grep`, each is a true operator, evaluating its block expression without an interposed function call frame. Thus any `caller` or `return` expression or similar within the block will directly affect the function containing the `any` or `all` expression itself. 39 | 40 | These operators only yield a single scalar; in list context therefore they will just provide a single-element list containing that boolean scalar. This is so that there are no "surprises" if the operator is used in a list context, such as when building a key/value pair list for the constructor of an object. By returning a single false value even as a list, rather than an empty list, such constructions do not cause issues. 41 | 42 | For example: 43 | 44 | ```perl 45 | Some::Class->new( 46 | option => (any { TEST } list, of, things), 47 | other => $parameter, 48 | ); 49 | ``` 50 | 51 | ## Backwards Compatibility 52 | 53 | As these new operators are guarded by named features, there are no immediate concerns with backward compatiblity in the short-term. 54 | 55 | In the longer term, if these named features become part of a versioned feature bundle that is enabled by a corresponding `use VERSION` declaration there may be concerns that the names collide with functions provided by `List::Util` or similar modules. As the intention of these operators is to provide the same behaviour, this is not considered a major problem. Differences due to caller scope as outlined above may be surprising to a small number of users. 56 | 57 | ## Security Implications 58 | 59 | ## Examples 60 | 61 | ```perl 62 | use v5.40; 63 | use feature 'any'; 64 | 65 | if( any { $_ > 10 } 5, 10, 15, 20 ) { say "A number above 10" } 66 | ``` 67 | 68 | ## Prototype Implementation 69 | 70 | The overall behaviour of these operators is primarily demonstrated by functions from core's existing [`List::Util`](https://metacpan.org/pod/List::Util) module. Additional examples of a more efficient keyword-and-operator implementation can be found in [`List::Keywords`](https://metacpan.org/pod/List::Keywords). 71 | 72 | ## Future Scope 73 | 74 | ### Named Lexicals 75 | 76 | The `List::Keywords` module also provides an interesting "named lexical" syntax to its operators, allowing the user to specify a lexical variable, rather than the global `$_`, to store each item for iteration: 77 | 78 | ```perl 79 | use List::Keywords qw( any ); 80 | 81 | if( any my $item { we_want($item) } @items ) { 82 | say "There's an item we want here"; 83 | } 84 | ``` 85 | 86 | These lexicals are useful when nesting multiple calls to list-processing operators, to avoid collisions in the use of the `$_` global, and lead to cleaner code. They are also useful for suggesting how to support n-at-a-time behaviour of `grep` and `map`-like functions. 87 | 88 | If this feature is to be considered, it will require careful thought on how it might interact with the so-far-unspecified idea of accepting `any EXPR, LIST` as `grep` currently does. I would recommend not allowing that variant, to allow for easier implementation of these named lexicals in future as they provide advantages that outweigh the minor inconvenience of having to wrap the expression in brace characters. 89 | 90 | ### Other Operators 91 | 92 | * The other two variant behaviours of `none` and `notall`. These simply invert the sense of the filter block. 93 | 94 | * Another variation on the theme, `first`. This returns the value from the list itself, that first caused the filter block to be true. 95 | 96 | ## Rejected Ideas 97 | 98 | ### Block-less syntax 99 | 100 | Supporting syntax analogous to the "deferred-expression" form of `grep EXPR, LIST`. 101 | 102 | ### Keywords as Junctions 103 | 104 | Using the `any` and `all` keywords to make junction-like behaviour. Such is already provided by other modules, for example [`Data::Checks`](https://metacpan.org/pod/Data::Checks) in a title-case form and thus would not collide with the all-lowercase keywords provided here. This is already possible: 105 | 106 | ```perl 107 | use Data::Checks qw( Any ... ); 108 | use Syntax::Operator::Is; 109 | 110 | if( $x is Any( things... ) ) { ... } 111 | ``` 112 | 113 | In any case, as junctions behave like values, they do not require special syntax like the block-invoking keywords proposed here, so they can be provided by regular function-call syntax from regular modules. 114 | 115 | ## Open Issues 116 | 117 | * There could be anything up to five new operators added by this idea. Do they all get their own named feature flags? Do they all live under one flag? 118 | 119 | * Should the flag be called `any`? That might be confusing as compared to the `:any` import tag which would request all features. 120 | 121 | ## Copyright 122 | 123 | Copyright (C) 2024, Paul Evans. 124 | 125 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 126 | -------------------------------------------------------------------------------- /ppcs/ppc0028-custom-prefix-postfix-operators.md: -------------------------------------------------------------------------------- 1 | # Custom Pre- and Post-fix Operators 2 | 3 | ## Preamble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0028 8 | Status: Exploratory 9 | 10 | ## Abstract 11 | 12 | Expand the current `PL_infix_plugin` and mechanism and `XS::Parse::Infix` to support prefix and postfix operators as well. 13 | 14 | ## Motivation 15 | 16 | Perl version 5.38 added a new internal mechanism, named `PL_infix_plugin`, which allows external modules to assist the parser to define new infix operators, either named like identifiers, or with sequences of non-identifier characters. This mechanism is exposed to XS module authors via the `XS::Parse::Infix` module, allowing new operator behaviours to be created. There are several on CPAN using this mechanism, mostly named within the `Syntax::Operator::*` space. 17 | 18 | While originally intended to allow CPAN modules to experiment with new kinds of comparison or matching operators, such as `equ`, `eqr` and `is`, the mechanism has also been used for other cases such as infix zip and mesh operations. It may further be the case that other new kinds of operators would be useful, if Perl allowed prefix and postfix operators, in addition to the current infix kind. 19 | 20 | Additionally it may also be useful to support nullary operators (i.e. operators that take no operand), as a minor extension of the idea. There is limited utility here in such ideas as being able to support a mathematical "pi" or infinity constant using Unicode symbols. 21 | 22 | ## Rationale 23 | 24 | (explain why the (following) proposed solution will solve it) 25 | 26 | ## Specification 27 | 28 | As most of the existing infrastructure is named specifically to do with infix operators, it would be relatively easy to add a new mechanism that can cover more shapes of operator. 29 | 30 | The existing components are: 31 | 32 | ``` 33 | PL_infix_plugin 34 | 35 | enum Perl_custom_infix_precedence { 36 | INFIX_PREC_LOW, 37 | INFIX_PREC_LOGICAL_OR_LOW, 38 | ... 39 | }; 40 | 41 | struct Perl_custom_infix { 42 | OP (*build_op)(pTHX_ SV **opdata, OP *lhs, OP *rhs, struct Perl_custom_infix *); 43 | } 44 | ``` 45 | 46 | They could be renamed to 47 | 48 | ``` 49 | PL_operator_plugin 50 | 51 | enum Perl_custom_operator_precedence { 52 | INFIX_PREC_LOW, 53 | ... 54 | PREFIX_LOW, 55 | ... 56 | POSTFIX_LOW, 57 | ... 58 | }; 59 | 60 | struct Perl_custom_operator { 61 | OP (*build_op)(pTHX_ SV **opdata, OP *lhs, OP *rhs, struct Perl_custom_operator *); 62 | /* `rhs` is ignored for non-infix operators */ 63 | } 64 | ``` 65 | 66 | There is however a possiblity of a conceptual clash with the other "custom operators" idea, which was added in Perl 5.14 and allows new kinds of opcodes in generated optrees whose behaviour is provided by XS functions. Those are not "operators" in the syntax sense of the word. 67 | 68 | Likewise, the `XS::Parse::Infix` module would get renamed to `XS::Parse::Operator`, and allow for expanded API to support these new operator shapes. 69 | 70 | ## Backwards Compatibility 71 | 72 | This proposal only intends to rename a core mechanism used by a special-purpose CPAN module, and does not directly alter any user-visible parts. That CPAN module (`XS::Parse::Infix`) would to be changed to support the new name and behaviour. A wrapper can be provided under its original name to support existing operator modules. 73 | 74 | ## Security Implications 75 | 76 | This proposal adds a minor extension to an existing parser mechanism that is only invoked during compiletime of program source. It is unlikely that any new issues are introduced by such an extension. 77 | 78 | ## Examples 79 | 80 | As this proposal intends to add a core interpreter mechanism for use by a module that is itself simply used by modules that provide syntax, it is hard to directly write any examples of its use. More illustrative may be to consider the kinds of new operator modules that such a mechanism would permit to be written. 81 | 82 | It is easily possible to come up with trivial "syntax neatening" of existing operators as mathematical notation, such as squaring and square-roots. 83 | 84 | ```perl 85 | use Syntax::Operator::Square; ## provides postfix ² 86 | use Syntax::Operator::SquareRoot; ## provides prefix √ 87 | 88 | my $hyp = √($x² + $y²); 89 | 90 | # equivalent to 91 | my $hyp = sqrt($x**2 + $y**2); 92 | ``` 93 | 94 | For another example, consider the use of a summation operator: 95 | 96 | ```perl 97 | use Syntax::Operator::Sum; 98 | 99 | my @numbers = ...; 100 | my $total = ∑ @numbers; 101 | ``` 102 | 103 | It is admittedly hard to see what this prefix operator notation gains over the far more traditional and widely-supported idea of simply having a `sum` function - such as that provided by `List::Util` - that operates on the array. 104 | 105 | I must admit, my imagination is somewhat lacking on some particularly motivating examples. It may be the case that there are genuinely-useful ideas for new operators that could be made that cannot currently be provided in existing syntax, but as yet I have failed to actually think of one. 106 | 107 | ## Prototype Implementation 108 | 109 | None is currently possible as this would need to be supported by the actual Perl parser as implemented in `perly.y`, `toke.c` and related files. If this proposal is accepted as exploratory, a prototype implementation can be directly implemented as a branch of the core perl interpreter. 110 | 111 | ## Future Scope 112 | 113 | * The `XS::Parse::Infix` module, or its more generically-named successor, could be considered for eventual inclusion in core perl directly, making it available to authors without needing to fetch it from CPAN. 114 | 115 | * Other more complex operator shapes - circumfix notation around an expression, or more interleaved shapes of syntax and operands. These would need to be considered on a per-usecase basis. 116 | 117 | ## Rejected Ideas 118 | 119 | ## Open Issues 120 | 121 | The entire thing? ;) As hinted above in the examples section, it isn't immediately clear that this mechanism is even required at all, which is why in my original implementation I restricted it to only infix operators. If there are specific motivating ideas of prefix or postfix operators that cannot be adequately expressed in the existing function-call notation then these would be useful to consider. 122 | 123 | ## Copyright 124 | 125 | Copyright (C) 2024, Paul Evans. 126 | 127 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 128 | -------------------------------------------------------------------------------- /ppcs/ppc0030-undef-aware-equality.md: -------------------------------------------------------------------------------- 1 | # `undef`-aware Equality Operators 2 | 3 | ## Preamble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0030 8 | Status: Exploratory 9 | 10 | ## Abstract 11 | 12 | Adds new infix comparison operators that are aware of the special nature of the `undef` value, comparing it as distinct from the number zero or the empty string. 13 | 14 | ## Motivation 15 | 16 | Perl has two sets of comparison operators; one set that considers the stringy nature of values, and another that considers numerical values. When comparing against `undef`, the stringy operators treat undef as an empty string, and the numerical operators treat it as zero. In both cases, a warning is produced: 17 | 18 | ``` 19 | Use of uninitialized value in string eq at ... 20 | ``` 21 | 22 | Sometimes it is useful to consider that `undef` is in fact a different value, distinct from any defined string or number - even empty or zero. Currently in Perl, to check if a given value (which may be `undef`) is equal to a given string value, the test must make sure to check for definedness: 23 | 24 | ```perl 25 | if( defined $x and $x eq $y ) { ... } 26 | ``` 27 | 28 | Furthermore, if the value `$y` can also be undefined, it may be desired that in that case it matches if `$x` is also undefined. In that situation, the two cases must be considered individually: 29 | 30 | ```perl 31 | if( (!defined $x and !defined $y) or 32 | (defined $x and defined $y and $x eq $y) ) { ... } 33 | ``` 34 | 35 | Countless bugs across countless modules have been caused by failing to check in such a way and using code such as simply `$x eq $y` in these circumstance. 36 | 37 | By providing new comparison operators that have different behaviour with undefined values, Perl can provide more convenient choice of behaviours for authors to use. 38 | 39 | ## Rationale 40 | 41 | (explain why the (following) proposed solution will solve it) 42 | 43 | ## Specification 44 | 45 | A new operator, named `equ`, which has the same syntax as regular string equality `eq`, but semantics identical to that given above: 46 | 47 | ```perl 48 | $x equ $y 49 | 50 | # Equivalent to (!defined $x and !defined $y) or 51 | # (defined $x and defined $y and $x eq $y) 52 | ``` 53 | 54 | In particular, given two `undef` values, this operator will yield true. Given one `undef` and one defined value, it yields false, or given two defined values it will yield the same answer that `eq` would. In no circumstance will it provoke a warning of undefined values. 55 | 56 | Likewise, a new operator named `===`, which provides the numerical counterpart: 57 | 58 | ```perl 59 | $x === $y 60 | 61 | # Equivalent to (!defined $x and !defined $y) or 62 | # (defined $x and defined $y and $x == $y) 63 | ``` 64 | 65 | Note that while the `===` operator will not provoke warnings about undefined values, it could still warn about strings that do not look like numbers. 66 | 67 | ## Backwards Compatibility 68 | 69 | As these infix operators are currently syntax errors in existing versions of Perl, it is not required that they be feature-guarded. 70 | 71 | However, it may still be considered useful to add a feature guard in order to provide some compatibility with existing code which uses the `Syntax::Operator::Equ` CPAN module. Likely this would be named `equ`: 72 | 73 | ```perl 74 | use feature 'equ'; 75 | ``` 76 | 77 | Once the feature becomes stable it is likely this would be included in an appropriate `use VERSION` bundle, so users would not be expected to request it specifically in the long-term. 78 | 79 | ## Security Implications 80 | 81 | No new issues are anticipated. 82 | 83 | ## Examples 84 | 85 | ## Prototype Implementation 86 | 87 | This operator is already implemented using the pluggable infix operator support of Perl version 5.38, in CPAN module [`Syntax::Operator::Equ`](https://metacpan.org/pod/Syntax::Operator::Equ). 88 | 89 | ## Future Scope 90 | 91 | Considering further the possible idea to provide a `match/case` syntax inspired by [`Syntax::Keyword::Match`](https://metacpan.org/pod/Syntax::Keyword::Match), this operator would be a useful addition in combination with that as well, allowing dispatch on a set of fixed values as well as `undef`: 92 | 93 | ```perl 94 | match( $x : equ ) { 95 | case(undef) { say "The value in x is undefined" } 96 | case("") { say "The value in x is empty string" } 97 | case("ABC") { say "The value in x is ABC" } 98 | ... 99 | } 100 | ``` 101 | 102 | ## Rejected Ideas 103 | 104 | * It is not possible to unambiguously provide extensions of the ordering operators `cmp` and `<=>` in a similar way, because this requires a decision to be made as to the sorting order of undefined values, compared to any defined string or number. While it may be natural to consider that undef sorts before an empty string, there is no clear choice on where undef would sort compared to (signed) numbers. Would it be before any number, even negative infinity? This would place it far away from zero. 105 | 106 | * Likewise, because it is not possible to provide an undef-aware version of `cmp` or `<=>`, the ordering comparison operators of `le`, `lt` and so on are also not provided. 107 | 108 | ## Open Issues 109 | 110 | * Should we also provide negated versions of these operators? While much rarer in practice, it may be useful to provide a "not equ", perhaps spelled `nequ` or `neu`; and likewise `!===` or `!==` for the numerical version. These do not suffer the sorting order problem outlined above for more general comparisons. 111 | 112 | * As an entirely alternate proposal, should we instead find ways to apply behaviour-modifying flags to the existing operators? That is, rather than adding a new `equ` and `===` could we instead consider some syntax such as `eq:u` and `==:u` as a modifier flag, similar to the flags on regexp patterns, as a way to modify operators? This would be extensible in a more general way to more operators, while also allowing more flexible flags in future, such as for instance a case-ignoring string comparison to be spelled `eq:i`. This alternate proposal is be the subject of an [alternate PPC document](ppc0031-metaoperator-flags.md). 113 | 114 | * How to pronounce the name of this new operator? I suggest "ee-koo", avoiding the "you" part of the sound. 115 | 116 | ## Copyright 117 | 118 | Copyright (C) 2024, Paul Evans. 119 | 120 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 121 | -------------------------------------------------------------------------------- /ppcs/ppc0031-metaoperator-flags.md: -------------------------------------------------------------------------------- 1 | # Meta-operator Flags to Modify Behaviour of Operators 2 | 3 | ## Preamble 4 | 5 | Author: Paul Evans 6 | Sponsor: 7 | ID: 0031 8 | Status: Exploratory 9 | 10 | ## Abstract 11 | 12 | Defines a new syntax for adding behaviour-modifying flags to operators, in order to extend the available set of behaviours. 13 | 14 | ## Motivation 15 | 16 | This idea came out of discussions around the proposal to add new `equ` and `===` operators, which are aware of the special nature of the `undef` value. The idea is that rather than adding just two new special-purpose operators, a far more flexible and extensible idea may be to allow syntax for putting flags on the existing operators. 17 | 18 | The central idea here is that existing operators can be modified by a set of single-letter suffix flags, that could be considered slightly similar to the concept of regexp pattern flags. 19 | 20 | ## Rationale 21 | 22 | (explain why the (following) proposed solution will solve it) 23 | 24 | ## Specification 25 | 26 | Following the name of a regular value-returning infix operator, an optional set of flag letters can be supplied after a separating colon: 27 | 28 | ```perl 29 | $x eq:u $y 30 | ``` 31 | 32 | In terms of syntax, these are flags that alter the runtime behaviour of the operator, and do not change the way it parses at compile-time. The particular behaviour being changed would depend on the operator and the flag applied, but in order to make it easy to learn and use, some consistency should be applied to the choice of flags. 33 | 34 | The `:u` flag alters the behaviour of a scalar-comparing operator (`eq`, `ne`, `==`, `!=`) in the same way described in the `equ` proposal; namely, that these operators now consider that `undef` is equal to another `undef` but unequal to any defined value, even the empty string or number zero. 35 | 36 | ```perl 37 | $x eq:u $y 38 | 39 | # Equivalent to (!defined $x and !defined $y) or 40 | # (defined $x and defined $y and $x eq $y) 41 | 42 | $x ==:u $y 43 | 44 | # Equivalent to (!defined $x and !defined $y) or 45 | # (defined $x and defined $y and $x == $y) 46 | ``` 47 | 48 | The `ne` and `!=` operators can likewise be modified to produce the negation of these ones. 49 | 50 | ```perl 51 | $x ne:u $y 52 | 53 | # Equivalent to not( $x eq:u $y ) 54 | # i.e. (defined $x xor defined $y) or 55 | # (defined $x and defined $y and $x ne $y) 56 | 57 | $x !=:u $y 58 | 59 | # Equivalent to not( $x ==:u $y ) 60 | # i.e. (defined $x xor defined $y) or 61 | # (defined $x and defined $y and $x != $y) 62 | ``` 63 | 64 | Another flag, `:i`, modifies the string comparison operator making it case-insensitive, acting as if its operands were first passed through the `fc()` function: 65 | 66 | ```perl 67 | $x eq:i $y 68 | 69 | # Equivalent to fc($x) eq fc($y) 70 | ``` 71 | 72 | There is no equivalent on a numerical comparison; attempting to apply the flag here would result in a compile-time error. 73 | 74 | Finally, note that these two flags can be combined, and in that case the order does not matter. 75 | 76 | ```perl 77 | $x eq:ui $y 78 | $x eq:iu $y 79 | 80 | # Equivalent to (!defined $x and !defined $y) or 81 | # (defined $x and defined $y and fc($x) eq fc($y)) 82 | ``` 83 | 84 | ## Backwards Compatibility 85 | 86 | As these notations are currently syntax errors in existing versions of Perl, it is not required that they be feature-guarded. 87 | 88 | In particular, there is no potential that a colon-prefixed identifier name is confused with a statement label, because a statement label must appear at the beginning of a statement, whereas an infix operator must appear after its left-hand operand: 89 | 90 | ```perl 91 | use v5.xx; 92 | 93 | eq:u $y; # A call to the u($x) function, in a statement labeled as 'eq:' 94 | 95 | $x eq:u $y; # An expression using the eq operator modified with :u 96 | ``` 97 | 98 | Similarly, there is no ambiguity with the ternary conditional operator pair `? :`, because the colon for these flags only appears when the right-hand operand to an infix operator is expected: 99 | 100 | ```perl 101 | use v5.xx; 102 | 103 | my $result = $stringy ? $x eq:u $y : $x ==:u $y; 104 | # Parser always knows these ^------------^ are meta-operator flags 105 | ``` 106 | 107 | Finally, there is no ambiguity with attribute syntax - even when considering extension 108 | proposals that would extend the attribute syntax to more sites - again because of the non-overlap between applicable situations. In fact it could be argued that these flags are a similar and related syntax; appearing to apply some sort of adverb-like "attribute" to an operator. Where full attributes have identifiers as names and can take optional values, these flags are bundled into single-letter names without options. 109 | 110 | ## Security Implications 111 | 112 | No new issues are anticipated. 113 | 114 | ## Examples 115 | 116 | A few examples are given in the specification above. 117 | 118 | ## Prototype Implementation 119 | 120 | It is currently not thought possible to prototype this exact syntax as a CPAN module, because it requires extending the parser in ways that neither the `PL_keyword_plugin` nor `PL_infix_plugin` extension mechanisms can achieve. 121 | 122 | If it were considered necessary to prototype these on CPAN in some manner, it would be possible to define some new operator names, such as `eqx` for "extensible `eq`", which would then allow such suffix flag notation on them. While it would then be possible to experiment with possible behaviours of new flag letters, it would remove most of the advantage of the idea in that it uses (and applies to) existing operator names, rather than inventing new ones. It is therefore unlikely to be worth performing such an experiment. 123 | 124 | ## Future Scope 125 | 126 | As this proposal introduces the idea of adding flags named by letters to existing Perl operators, there is clearly much more potential scope to define other modifier letters to these or other operators. 127 | 128 | However, care should definitely be taken to limit these new definitions to genuinely useful combinations that could not easily be achieved by other means. With over 30 infix operators and 52 single-letter flags available, the number of possible ideas far exceeds the number of combinations that would actually be useful in real-world situations, and sufficiently motivating to justify adding more things for users to learn and recognise. 129 | 130 | In particular, care should be taken that any new flag proposals do not attempt to reuse existing flag letters to have other meanings. For example, if the `:i` flag is added to other operators its meaning should be somehow compatible with the "ignore-case" of `eq`, rather than serve some other unrelated purpose. 131 | 132 | ## Rejected Ideas 133 | 134 | ### Using the `/` symbol as a flag separator 135 | 136 | It may look similar to the regexp pattern flags and suggest a similar purpose, but it becomes more of a problem to suffix attach these onto existing numerical operators. While not suggested by this document, consider the hypothetical case of adding a flag to the division operator such as `z` to alter how it behaves on division-by-zero. The `:` symbol allows this as `/:z` whereas the `/` symbol leads to the problematic `//z` operator. There are no existing infix operators that use the `:` symbol. 137 | 138 | ## Open Issues 139 | 140 | ### Is this use of the `:` symbol still too much? 141 | 142 | Considered alone in this proposal, the syntax is unambiguous and makes sense with the existing Perl syntax. In particular, the similarity with attributes is noted. However, when considering such other possible ideas as the `in:OP` hyper-operator, or the `match/case` syntax to replace the problematic `given/when` and smartmatch, there seems to be an explosion in the possible meanings of the colon character: 143 | 144 | ```perl 145 | if( $str in:eq:u @possible_matches ) { ... } 146 | # ^--^ these two colons mean conceptually different things 147 | ``` 148 | 149 | ```perl 150 | match( $value : eq:u ) { 151 | # ^---^ these two colons also mean conceptually different things 152 | case(undef) { say "It was undefined" } 153 | case("") { say "It was empty string" } 154 | default { say "It was a non-empty string" } 155 | } 156 | ``` 157 | 158 | In each case here the Perl parser (or another other static analysis tooling, such as syntax highlighters) should have no trouble understanding the various meanings. However, it may cause some confusion to human readers as to what the different uses all mean. I have tried to be consistent with the use of whitespace before or after the colon symbol to hint at its various different meanings in the examples above. These are currently purely informational hints to human readers, and not considered significant by the Perl parser. 159 | 160 | There aren't many other viable choices of symbol, at least not while remaining both within the character set provided by ASCII, and not being sensitive to the presence of whitespace. This may be the inevitable pressure of trying to add more features to such an operator-rich language as Perl, while attempting to stick to ASCII and whitespace-agnostic syntax. At some point we may have to accept that we cannot add new operator-based syntax without either accepting the use of non-ASCII Unicode operators, or using presence-of-whitespace hints to disambiguate different cases. 161 | 162 | ## Copyright 163 | 164 | Copyright (C) 2024, Paul Evans. 165 | 166 | This document and code and documentation within it may be used, redistributed and/or modified under the same terms as Perl itself. 167 | -------------------------------------------------------------------------------- /ppcs/ppc0033-ampersand-method-calls.md: -------------------------------------------------------------------------------- 1 | # Allow calling subs using method-like syntax 2 | 3 | ## Preamble 4 | 5 | Author: Graham Knop 6 | ID: 0033 7 | Status: Implemented 8 | 9 | ## Abstract 10 | 11 | Add a syntax for calling lexical methods that is similar to the existing 12 | method syntax, rather than needing to call them like subs. The new syntax 13 | `$object->&method(@args)` would be equivalent to sub call to `method`, but 14 | using a syntax closer to method calls. 15 | 16 | ## Motivation 17 | 18 | When using a lexical sub inside an object class, the subs must be called as 19 | `method($object, @args)`. This works, but for many people, this feels wrong 20 | as a way to call something thought of as a private method. This has resulted 21 | in some people using syntax like `$object->${\&method}(@args)`, or continuing 22 | to use code refs stored in a lexical variable (`$object->$method(@args)`). 23 | 24 | ## Rationale 25 | 26 | `$object->method(@args)` always does a method lookup, ignoring the local 27 | context. Changing this would result in a lot of broken code. Changing this 28 | only for new code (via feature or other lexical effect) would introduce 29 | confusion and would also require some new (or pessimised) syntax to allow 30 | the previous behavior. 31 | 32 | `$object->&method(@args)` would be an unambiguous way to call the `method` in 33 | the current scope, whether a lexical or package sub. 34 | 35 | A truly private method does not need to do any lookup via `@ISA`, so being 36 | equivalent to a sub call makes sense. 37 | 38 | This also would pair well with 39 | [PPC0021](ppc0021-optional-chaining-operator.md) to allow optional sub calls 40 | as part of a chain. 41 | 42 | ## Specification 43 | 44 | `$object->&method(@args)` would behave exactly the same as `&method($object, 45 | @args)`. `method` would be searched for in the current lexical scope. If not 46 | found, it would be looked up in the current package. If still not found, the 47 | current package's `AUTOLOAD` would be called. If `AUTOLOAD` does not exist, an 48 | error would be issued. The call would ignore prototypes, just as traditional 49 | method calls and `&subroutine()` calls do. 50 | 51 | Methods defined using the `class` syntax would also be callable using this 52 | syntax. 53 | 54 | ## Backwards Compatibility 55 | 56 | The syntax `->&word` is currently a syntax error, so no backwards 57 | compatibility issues are forseen. 58 | 59 | ## Security Implications 60 | 61 | As this is purely a syntax feature that is currently a syntax error, no 62 | security implications are forseen. 63 | 64 | ## Examples 65 | 66 | ```perl 67 | use v5.36; 68 | 69 | package Foo { 70 | sub new ($class) { 71 | return bless { 72 | field => 5, 73 | }, $class; 74 | } 75 | 76 | my sub private ($self) { 77 | return $self->{field}; 78 | } 79 | 80 | sub _old_school_private ($self) { 81 | return $self->{field}; 82 | } 83 | 84 | sub public ($self) { 85 | 86 | # call via lexical method is allowed 87 | say "my field: " . $self->&private; 88 | # exactly equivalent to: 89 | say "my field: " . private($self); 90 | 91 | 92 | # can also be used to call package methods. But won't look up via @ISA 93 | say "my field: " . $self->&_old_school_private; 94 | # exactly equivalent to: 95 | say "my field: " . _old_school_private($self); 96 | } 97 | } 98 | 99 | my $foo = Foo->new; 100 | $foo->public; 101 | 102 | # this is an error, as there is no sub named "private" in scope 103 | $foo->&private; 104 | 105 | # this is also an error, as "public" does not exist in the current scope 106 | $foo->&public; 107 | ``` 108 | 109 | ## Prototype Implementation 110 | 111 | * [Object::Pad::LexicalMethods](https://metacpan.org/pod/Object::Pad::LexicalMethods) 112 | implements lexical methods and `->&` method calls for 113 | [Object::Pad](https://metacpan.org/pod/Object::Pad). 114 | 115 | ## Future Scope 116 | 117 | Lexical methods (`my method foo`) are not currently supported under the 118 | `class` syntax. If they were added, `->&method` would be the preferred way to 119 | call them. 120 | 121 | ## Rejected Ideas 122 | 123 | Allowing `$object->method` to call lexical subs would break existing code and 124 | would interfere with the ability to call the actual method. Perl doesn't know 125 | what type `$object` is, so it can't know when it is appropriate to do method 126 | lookup and when to call a local sub. Using a different syntax at the call site 127 | avoids these issues. 128 | 129 | ## Open Issues 130 | 131 | None currently. 132 | 133 | ## Copyright 134 | 135 | Copyright (C) 2025, Graham Knop 136 | 137 | This document and code and documentation within it may be used, redistributed 138 | and/or modified under the same terms as Perl itself. 139 | -------------------------------------------------------------------------------- /ttlib/page.tt: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | [% IF base -%] 5 | 6 | [% END -%] 7 | 8 | 9 | 10 | 11 | 12 | Proposed Perl Changes 13 | 14 | 15 | 16 | 17 | 18 | 19 | 23 | 24 | 39 | 40 | 41 | 43 | 45 | 46 | 47 | 48 | 49 | 50 | 51 |
52 | 53 | 85 |
86 | 87 | 88 |
89 |
90 | [% content %] 91 |
92 |
93 | 94 | 100 | 101 | 104 | 105 | 106 | --------------------------------------------------------------------------------