├── .gitignore ├── .rspec ├── .travis.yml ├── CHANGELOG.md ├── Gemfile ├── LICENSE.txt ├── README.md ├── Rakefile ├── bin ├── console └── setup ├── copy-help-text ├── exe └── rexe ├── rexe.gemspec ├── sample-data └── eur_rates.json └── spec ├── dummy.rb ├── rexe_spec.rb └── spec_helper.rb /.gitignore: -------------------------------------------------------------------------------- 1 | /.bundle/ 2 | /.yardoc 3 | /_yardoc/ 4 | /coverage/ 5 | /doc/ 6 | /pkg/ 7 | /spec/reports/ 8 | /tmp/ 9 | /.idea/ 10 | 11 | # rspec failure tracking 12 | .rspec_status 13 | 14 | Gemfile.lock 15 | projectFilesBackup/ 16 | *gem 17 | -------------------------------------------------------------------------------- /.rspec: -------------------------------------------------------------------------------- 1 | --format documentation 2 | --color 3 | --require spec_helper 4 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | --- 2 | sudo: false 3 | language: ruby 4 | cache: bundler 5 | rvm: 6 | - 2.6.0 7 | before_install: gem install bundler -v 2.0.1 8 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | ## rexe -- Ruby Command Line Executor/Filter 2 | 3 | ### 1.7.0 4 | 5 | * Replace amazing_print dependency with awesome_print. 6 | * Effectively disable `try` method, i.e. print stack trace on errors. 7 | * In `try` method, rescue StandardError instead of Exception. 8 | 9 | 10 | ### 1.6.0 11 | 12 | * Replace calls to `exists?`, which has been removed, with `exist?`. 13 | * Add warning messages to stderr for unspecified options (input/output format, input mode), 14 | but only when logging is disabled. 15 | 16 | ### 1.5.1 17 | 18 | * Change string concatenations using << to creation of new strings with += to accommodate default-frozen strings. 19 | 20 | 21 | ### 1.5.0 22 | 23 | * Switch from AwesomePrint to AmazingPrint, and change all references in text, help, etc. 24 | 25 | 26 | ### 1.4.1 27 | 28 | * As of later versions of Ruby, `stringio` is a gem and must be required. Fixed. 29 | 30 | 31 | ### 1.4.0 32 | 33 | * Handle Bundler deprecation of `with_clean_env` method. 34 | * Mark entire file frozen_string_literal: true. 35 | 36 | 37 | ### 1.3.1 38 | 39 | * Add Awesome Print and Pretty Print output formats to help. 40 | 41 | 42 | ### 1.3.0 43 | 44 | * Document --project-url option. 45 | * Add undocumented option --open-project to output Github project URL. 46 | * Simplify test as per @davetron5000's array based approach. 47 | * Fix context do/end in test code. 48 | * Froze strings: VERSION, PROJECT_URL, help_text. 49 | 50 | 51 | ### 1.2.0 52 | 53 | * Add --project-url option to output project URL on Github, then exit 54 | 55 | 56 | ### 1.1.0 57 | 58 | * Enable specifying different output formats for tty and block devices. (#4) 59 | * Outputs exception text on error instead of just exception class. 60 | 61 | 62 | ### 1.0.3 63 | 64 | * Fix/improve help text. 65 | 66 | 67 | ### 1.0.2 68 | 69 | * Add mention of -v/--version to help text. 70 | 71 | 72 | ### 1.0.1 73 | 74 | * Improve help text. 75 | * Improve code block display in readme. 76 | 77 | 78 | ### 1.0.0 79 | 80 | * Suppress help message on SystemExit. 81 | * Eliminate stack trace from all error messages. 82 | * Fix Awesome Print output to have a "\n" at the end of it. 83 | * Add eur_rates.json sample data. 84 | * Remove gem post-commit message. 85 | 86 | 87 | ### 0.15.1 88 | 89 | * Fix help text in readme. 90 | 91 | 92 | ### 0.15.0 93 | 94 | * Source code now defaults to 'self' (#3). 95 | * Change parse errors to not output help text and stack trace, but instead a short message with suggestion to use -h. 96 | 97 | 98 | ### 0.14.0 99 | 100 | * The default output format has been changed from -op (:puts) to -on (:none). 101 | * Support automatic input from file with -f option. 102 | * Normalize load filespecs to eliminate duplication and to facilitate correct deletion. 103 | 104 | 105 | ### v0.13.0 106 | 107 | * Much refactoring. 108 | * Allow omitting source code in no-op mode. 109 | * Add ability to remove load or require files using minus sign preceding name. 110 | * Requires needed for parsing and formatting will now be included in log output. 111 | * Change license from MIT to Apache version 2. 112 | * Add undocumented '--open-project' command line option to launch Github project page in Mac OS and possibly other OS's. 113 | * Fix and add tests. 114 | 115 | 116 | ### v0.12.0 117 | 118 | * Change verbose -v boolean option to log output format -g option. 119 | * Print error message and exit with nonzero exit code if no source code provided. 120 | * Add $RC.i alias for $RC.count. 121 | 122 | 123 | ### v0.11.0 124 | 125 | * Make global $RC (Rexe Context) OpenStruct available to user code; added `count` for record count in `-ml` mode. 126 | * Change verbose output to YAML format. 127 | * Failure to load a file now raises an error. 128 | * Add pretty print output format. 129 | * Fix tests. 130 | 131 | 132 | ### v0.10.3 133 | 134 | * Fix: parsing should not be attempted if in no input mode. 135 | * Improve README. 136 | 137 | ### v0.10.2 138 | 139 | * Fix problem in :none input format mode. 140 | 141 | 142 | ### v0.10.1 143 | 144 | * Fix help text for input formats. 145 | 146 | 147 | ### v0.10.0 148 | 149 | * Add input format option -i to simplify ingesting JSON, YAML, Marshal formats. 150 | * Add Marshal output option. 151 | * In -mn mode: 152 | * change output behavior; now outputs last evaluated value like other modes. 153 | * self is now a newly created object (Object.new), providing a clean slate for adding instance variables, methods, etc. 154 | * Wrap execution in Bundler.with_clean_env to enable loading of gems not in Gemfile. 155 | 156 | 157 | ### v0.9.0 158 | 159 | * Change -ms (single or separate string) mode to -ml (line) mode. 160 | * Use article text for readme. 161 | 162 | 163 | ### v0.8.1 164 | 165 | * Fix and improve help text. 166 | 167 | 168 | ### v0.8.0 169 | 170 | * Add no-op mode to suppress execution of code (useful with -v). 171 | * Add clear mode to clear all options specified up to that point (useful to ignore REXE_OPTIONS environment variable settings). 172 | 173 | ### v0.7.0 174 | 175 | * Remove -u option to address issue #1. 176 | 177 | 178 | ### v0.6.1 179 | 180 | * Improve handling of nonexistent load files (-l, -u options). 181 | 182 | 183 | ### v0.6.0 184 | 185 | * Change default input mode from :string to :no_input. 186 | * Improve readme. 187 | * Add post install message warning about change of default input mode. 188 | 189 | 190 | ### v0.5.0 191 | 192 | * Add '!' to require and load command options to clear respective file lists. 193 | * In 'no input' mode (-mn), fix so that only output explicitly sent to stdout is output (unlike other modes). 194 | * Gemspec now parses version from text instead of loading the script. 195 | * Add tests. 196 | 197 | ### v0.4.1 198 | 199 | * Fix -r (require) bug. 200 | 201 | 202 | ### v0.4.0 203 | 204 | * Add -u option for loading files at current directory or above. 205 | * Fix command line option handling for disabling verbose mode previously enabled. 206 | * Improve README. 207 | 208 | 209 | ### v0.3.1 210 | 211 | * Help text fixes. 212 | 213 | 214 | ### v0.3.0 215 | 216 | * For consistency with requires, specifying multiple load files on the command line can be done 217 | with comma separated filespecs. 218 | 219 | 220 | ### v0.2.0 221 | 222 | * Improve README and verbose logging. 223 | 224 | ### v0.1.0 225 | 226 | * Add ability to handle input as a single multiline string (using -mb option). 227 | * Add -mn mode for no input at all. 228 | * Fix and improve usage examples in README. 229 | 230 | 231 | ### v0.0.2 232 | 233 | * Fix running-as-script test. 234 | 235 | 236 | ### v0.0.1 237 | 238 | * Initial version. 239 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source "https://rubygems.org" 2 | 3 | # Specify your gem's dependencies in rexe.gemspec 4 | gemspec 5 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright 2019 Bennett Business Solutions, Inc. 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Rexe 2 | 3 | __Rexe__ is a Ruby script and gem that multiplies Ruby's usefulness and conciseness on the command line by: 4 | 5 | * automating parsing and formatting using JSON, YAML, Ruby marshalling, Amazing Print, and others 6 | * simplifying the use of Ruby as a shell filter, optionally predigesting input as lines, an enumerator, or one big string 7 | * extracting the plumbing from the command line; requires and other options can be set in an environment variable 8 | * enabling the loading of Ruby helper files to keep your code DRY and your command line code high level 9 | * reading and evaluating a ~/.rexerc file on startup for your shared custom code and common requires 10 | 11 | ---- 12 | 13 | Shell scripting is great for simple tasks but for anything nontrivial it can easily get cryptic and awkward (pun intended!). 14 | 15 | This problem can often be solved by writing a Ruby script instead. Ruby provides fine grained control in a language that is all about clarity, conciseness, and expressiveness. 16 | 17 | Unfortunately, when there are multiple OS commands to be called, then Ruby can be awkward too. 18 | 19 | Sometimes a good solution is to combine Ruby and shell scripting on the same command line. Rexe multiplies your power to do so. 20 | 21 | 22 | ### Using the Ruby Interpreter on the Command Line 23 | 24 | Let's start by seeing what the Ruby interpreter already provides. Here we use `ruby` on the command line, using an intermediate environment variable to simplify the logic and save the data for use by future commands. An excerpt of the output follows the code: 25 | 26 | ```bash 27 | $ export EUR_RATES_JSON=`curl https://api.exchangeratesapi.io/latest` 28 | $ echo $EUR_RATES_JSON | ruby -r json -r yaml -e 'puts JSON.parse(STDIN.read).to_yaml' 29 | ``` 30 | ```yaml 31 | --- 32 | rates: 33 | MXN: 21.96 34 | AUD: 1.5964 35 | HKD: 8.8092 36 | ... 37 | base: EUR 38 | date: '2019-03-08' 39 | ``` 40 | 41 | Unfortunately, the configuration setup (the `require`s) along with the reading, parsing, and formatting make the command long and tedious, discouraging this approach. 42 | 43 | ### Rexe 44 | 45 | Rexe [see footnote ^1 regarding its origin] can simplify such commands. Among other things, rexe provides switch-activated input parsing and output formatting so that converting from one format to another is trivial. The previous `ruby` command can be expressed in `rexe` as: 46 | 47 | ```bash 48 | $ echo $EUR_RATES_JSON | rexe -mb -ij -oy self 49 | ``` 50 | 51 | Or, even more concisely (`self` is the default Ruby source code for rexe commands): 52 | 53 | ```bash 54 | $ echo $EUR_RATES_JSON | rexe -mb -ij -oy 55 | ``` 56 | 57 | The command options may seem cryptic, but they're logical so it shouldn't take long to learn them: 58 | 59 | * `-mb` - __mode__ to consume all standard input as a single __big__ string 60 | * `-ij` - parse that __input__ with __JSON__; `self` will be the parsed object 61 | * `-oy` - __output__ the final value as __YAML__ 62 | 63 | If input comes from a JSON or YAML file, rexe determines the input format from the file's extension, and it's even simpler: 64 | 65 | ```bash 66 | $ rexe -f eur_rates.json -oy 67 | ``` 68 | 69 | Rexe is at https://github.com/keithrbennett/rexe and can be installed with `gem install rexe`. Rexe provides several ways to simplify Ruby on the command line, tipping the scale so that it is practical to do it more often. 70 | 71 | ---- 72 | 73 | Here is rexe's help text as of the time of this writing: 74 | 75 | ``` 76 | rexe -- Ruby Command Line Executor/Filter -- v1.7.0 -- https://github.com/keithrbennett/rexe 77 | 78 | Executes Ruby code on the command line, 79 | optionally automating management of standard input and standard output, 80 | and optionally parsing input and formatting output with YAML, JSON, etc. 81 | 82 | rexe [options] [Ruby source code] 83 | 84 | Options: 85 | 86 | -c --clear_options Clear all previous command line options specified up to now 87 | -f --input_file Use this file instead of stdin for preprocessed input; 88 | if filespec has a YAML and JSON file extension, 89 | sets input format accordingly and sets input mode to -mb 90 | -g --log_format FORMAT Log format, logs to stderr, defaults to -gn (none) 91 | (see -o for format options) 92 | -h, --help Print help and exit 93 | -i, --input_format FORMAT Input format, defaults to -in (None) 94 | -ij JSON 95 | -im Marshal 96 | -in None (default) 97 | -iy YAML 98 | -l, --load RUBY_FILE(S) Ruby file(s) to load, comma separated; 99 | ! to clear all, or precede a name with '-' to remove 100 | -m, --input_mode MODE Input preprocessing mode (determines what `self` will be) 101 | defaults to -mn (none) 102 | -ml line; each line is ingested as a separate string 103 | -me enumerator (each_line on STDIN or File) 104 | -mb big string; all lines combined into one string 105 | -mn none (default); no input preprocessing; 106 | self is an Object.new 107 | -n, --[no-]noop Do not execute the code (useful with -g); 108 | For true: yes, true, y, +; for false: no, false, n 109 | -o, --output_format FORMAT Output format, defaults to -on (no output): 110 | -oa Amazing Print 111 | -oi Inspect 112 | -oj JSON 113 | -oJ Pretty JSON 114 | -om Marshal 115 | -on No Output (default) 116 | -op Puts 117 | -oP Pretty Print 118 | -os to_s 119 | -oy YAML 120 | If 2 letters are provided, 1st is for tty devices, 2nd for block 121 | --project-url Outputs project URL on Github, then exits 122 | -r, --require REQUIRE(S) Gems and built-in libraries to require, comma separated; 123 | ! to clear all, or precede a name with '-' to remove 124 | -v, --version Prints version and exits 125 | 126 | --------------------------------------------------------------------------------------- 127 | 128 | In many cases you will need to enclose your source code in single or double quotes. 129 | 130 | If source code is not specified, it will default to 'self', 131 | which is most likely useful only in a filter mode (-ml, -me, -mb). 132 | 133 | If there is a .rexerc file in your home directory, it will be run as Ruby code 134 | before processing the input. 135 | 136 | If there is a REXE_OPTIONS environment variable, its content will be prepended 137 | to the command line so that you can specify options implicitly 138 | (e.g. `export REXE_OPTIONS="-r awesome_print,yaml"`) 139 | ``` 140 | 141 | ### Simplifying the Rexe Invocation 142 | 143 | There are two main ways we can make the rexe command line even more concise: 144 | 145 | * by extracting configuration into the `REXE_OPTIONS` environment variable 146 | * by extracting low level and/or shared code into helper files that are loaded using `-l`, 147 | or implicitly with `~/.rexerc` 148 | 149 | 150 | ### The REXE_OPTIONS Environment Variable 151 | 152 | The `REXE_OPTIONS` environment variable can contain command line options that would otherwise be specified on the rexe command line: 153 | 154 | Instead of this: 155 | 156 | ```bash 157 | $ rexe -r wifi-wand -oa WifiWand::MacOsModel.new.wifi_info 158 | ``` 159 | 160 | you can do this: 161 | 162 | ```bash 163 | $ export REXE_OPTIONS="-r wifi-wand -oa" 164 | $ rexe WifiWand::MacOsModel.new.wifi_info 165 | $ # [more rexe commands with the same options] 166 | ``` 167 | 168 | Putting configuration options in `REXE_OPTIONS` effectively creates custom defaults, and is useful when you use the same options in most or all of your commands. Any options specified on the rexe command line will override the environment variable options. 169 | 170 | Like any environment variable, `REXE_OPTIONS` could also be set in your startup script, input on a command line using `export`, or in another script loaded with `source` or `.`. 171 | 172 | ### Loading Files 173 | 174 | The environment variable approach works well for command line _options_, but what if we want to specify Ruby _code_ (e.g. methods) that can be used by your rexe code? 175 | 176 | For this, rexe lets you _load_ Ruby files, using the `-l` option, or implicitly (without your specifying it) in the case of the `~/.rexerc` file. Here is an example of something you might include in such a file: 177 | 178 | ```ruby 179 | # Open YouTube to Wagner's "Ride of the Valkyries" 180 | def valkyries 181 | `open "http://www.youtube.com/watch?v=P73Z6291Pt8&t=0m28s"` 182 | end 183 | ``` 184 | 185 | To digress a bit, why would you want this? You might want to be able to go to another room until a long job completes, and be notified when it is done. The `valkyries` method will launch a browser window pointed to Richard Wagner's "Ride of the Valkyries" starting at a lively point in the music [see footnote ^2 regarding autoplay]. (The `open` command is Mac specific and could be replaced with `start` on Windows, a browser command name, etc.) [see footnote ^3 regarding OS portability]. 186 | 187 | If you like this kind of audio notification, you could download public domain audio files and use a command like player like `afplay` on Mac OS, or `mpg123` or `ogg123` on Linux. This approach is lighter weight, requires no network access, and will not leave an open browser window for you to close. 188 | 189 | Here is an example of how you might use the `valkyries` method, assuming the above configuration is loaded from your `~/.rexerc` file or an explicitly loaded file: 190 | 191 | ```bash 192 | $ tar czf /tmp/my-whole-user-space.tar.gz ~ ; rexe valkyries 193 | ``` 194 | 195 | (Note that `;` is used rather than `&&` because we want to hear the music whether or not the command succeeds.) 196 | 197 | You might be thinking that creating an alias or a minimal shell script (instead of a Ruby script) for this `open` would be a simpler and more natural approach, and I would agree with you. However, over time the number of these could become unmanageable, whereas using Ruby you could build a pretty extensive and well organized library of functionality. Moreover, that functionality could be made available to _all_ your Ruby code (for example, by putting it in a gem), and not just command line one liners. 198 | 199 | For example, you could have something like this in a gem or loaded file: 200 | 201 | ```ruby 202 | def play(piece_code) 203 | pieces = { 204 | hallelujah: "https://www.youtube.com/watch?v=IUZEtVbJT5c&t=0m20s", 205 | valkyries: "http://www.youtube.com/watch?v=P73Z6291Pt8&t=0m28s", 206 | wm_tell: "https://www.youtube.com/watch?v=j3T8-aeOrbg&t=0m1s", 207 | # ... and many, many more 208 | } 209 | `open #{Shellwords.escape(pieces.fetch(piece_code))}` 210 | end 211 | ``` 212 | 213 | ...which you could then call like this: 214 | 215 | ```bash 216 | $ tar czf /tmp/my-whole-user-space.tar.gz ~ ; rexe 'play(:hallelujah)' 217 | ``` 218 | 219 | (You need to quote the `play` call because otherwise the shell will process and remove the parentheses. Alternatively you could escape the parentheses with backslashes.) 220 | 221 | One of the examples at the end of this articles shows how you could have different music play for success and failure. 222 | 223 | 224 | ### Logging 225 | 226 | A log entry is optionally output to standard error after completion of the code. This entry is a hash representation (to be precise, `to_h`) of the `$RC` OpenStruct described in the $RC section below. It contains the version, date/time of execution, source code to be evaluated, options (after parsing both the `REXE_OPTIONS` environment variable and the command line), and the execution time of your Ruby code: 227 | 228 | ```bash 229 | $ echo $EUR_RATES_JSON | rexe -gy -ij -mb -oa -n self 230 | ``` 231 | ```yaml 232 | --- 233 | :count: 0 234 | :rexe_version: 1.3.1 235 | :start_time: '2019-09-11T13:28:46+07:00' 236 | :source_code: self 237 | :options: 238 | :input_filespec: 239 | :input_format: :json 240 | :input_mode: :one_big_string 241 | :loads: [] 242 | :output_format: :awesome_print 243 | :output_format_tty: :awesome_print 244 | :output_format_block: :awesome_print 245 | :requires: 246 | - awesome_print 247 | - json 248 | - yaml 249 | :log_format: :yaml 250 | :noop: true 251 | :duration_secs: 0.095705 252 | ``` 253 | 254 | We specified `-gy` for YAML format; there are other formats as well (see the help output or this document) and the default is `-gn`, which means don't output the log entry at all. 255 | 256 | The requires you see were not explicitly specified but were automatically added because Rexe will add any requires needed for automatic parsing and formatting, and we specified those formats in the command line options `-gy -ij -oa`. 257 | 258 | This extra output is sent to standard error (_stderr_) instead of standard output (_stdout_) so that it will not pollute the "real" data when stdout is piped to another command. 259 | 260 | If you would like to append this informational output to a file(e.g. `rexe.log`), you could do something like this: 261 | 262 | ```bash 263 | $ rexe ... -gy 2>>rexe.log 264 | ``` 265 | 266 | 267 | ### Input Modes 268 | 269 | Rexe tries to make it simple and convenient for you to handle standard input, and in different ways. Here is the help text relating to input modes: 270 | 271 | ``` 272 | -m, --input_mode MODE Input preprocessing mode (determines what `self` will be) 273 | defaults to -mn (none) 274 | -ml line; each line is ingested as a separate string 275 | -me enumerator (each_line on STDIN or File) 276 | -mb big string; all lines combined into one string 277 | -mn none (default); no input preprocessing; 278 | self is an Object.new 279 | ``` 280 | 281 | The first three are _filter_ modes; they make standard input available to your code as `self`. 282 | 283 | The last (and default) is the _executor_ mode. It merely assists you in executing the code you provide without any special implicit handling of standard input. Here is more detail on these modes: 284 | 285 | 286 | #### -ml "Line" Filter Mode 287 | 288 | In this mode, your code would be called once per line of input, and in each call, `self` would evaluate to each line of text: 289 | 290 | ```bash 291 | $ echo "hello\ngoodbye" | rexe -ml puts reverse 292 | olleh 293 | eybdoog 294 | ``` 295 | 296 | `reverse` is implicitly called on each line of standard input. `self` is the input line in each call (we could also have used `self.reverse` but the `self.` would have been redundant). 297 | 298 | Be aware that, in this mode, if you are using an automatic output mode (anything other than the default `-on` no output mode), although you can control the _content_ of output records, there is no way to selectively _exclude_ records from being output. Even if the result of the code is nil or the empty string, a newline will be output. To prevent this, you can do one of the following: 299 | 300 | * use `-me` Enumerator mode instead and call `select`, `filter`, `reject`, etc. 301 | * use the (default) `-on` _no output_ mode and call `puts` explicitly for the output you _do_ want 302 | 303 | 304 | #### -me "Enumerator" Filter Mode 305 | 306 | In this mode, your code is called only once, and `self` is an enumerator dispensing all lines of standard input. To be more precise, it is the enumerator returned by the `each_line` method, on `$stdin` or the input file, whichever is applicable. 307 | 308 | Dealing with input as an enumerator enables you to use the wealth of `Enumerable` methods such as `select`, `to_a`, `map`, etc. 309 | 310 | Here is an example of using `-me` to add line numbers to the first 3 files in the directory listing: 311 | 312 | ```bash 313 | $ ls / | rexe -me "first(3).each_with_index { |ln,i| puts '%5d %s' % [i, ln] }" 314 | 315 | 0 AndroidStudioProjects 316 | 1 Applications 317 | 2 Desktop 318 | ``` 319 | 320 | Since `self` is an enumerable, we can call `first` on it. We've used the default output mode `-on` (_no output_ mode), which says don't do any automatic output, just the output explicitly specified by `puts` in the source code. 321 | 322 | 323 | #### -mb "Big String" Filter Mode 324 | 325 | In this mode, all standard input is combined into a single (possibly large and possibly multiline) string. 326 | 327 | A good example of when you would use this is when you need to parse a multiline JSON or YAML representation of an object; you need to pass all the standard input to the parse method. This is the mode that was used in the first rexe example in this article. 328 | 329 | 330 | #### -mn "No Input" Executor Mode -- The Default 331 | 332 | In this mode, no special handling of standard input is done at all; if you want standard input you need to code it yourself (e.g. with `STDIN.read`). 333 | 334 | `self` evaluates to a new instance of `Object`, which would be used if you defined methods, constants, instance variables, etc., in your code. 335 | 336 | 337 | #### Filter Input Mode Memory Considerations 338 | 339 | If you are using one of the filter modes, and may have more input than would fit in memory, you can do one of the following: 340 | 341 | * use `-ml` (line) mode so you are fed only 1 line at a time 342 | * use an Enumerator, either by a) specifying the `-me` (enumerator) mode option, 343 | or b) using `-mn` (no input) mode in conjunction with something like `STDIN.each_line`. Then: 344 | * Make sure not to call any methods (e.g. `map`, `select`) 345 | that will produce an array of all the input because that will pull all the records into memory, or: 346 | * use [lazy enumerators](https://www.honeybadger.io/blog/using-lazy-enumerators-to-work-with-large-files-in-ruby/) 347 | 348 | 349 | ### Input Formats 350 | 351 | Rexe can parse your input in any of several formats if you like. You would request this in the _input format_ (`-i`) option. Legal values are: 352 | 353 | * `-ij` - JSON 354 | * `-im` - Marshal 355 | * `-in` - [None] (default) 356 | * `-iy` - YAML 357 | 358 | Except for `-in`, which passes the text to your code untouched, your input will be parsed in the specified format, and the resulting object passed into your code as `self`. 359 | 360 | The input format option is ignored if the input _mode_ is `-mn` ("no input" executor mode, the default), since there is no preprocessing of standard input in that mode. 361 | 362 | ### Output Formats 363 | 364 | Several output formats are provided for your convenience: 365 | 366 | * `-oa` - Amazing Print - calls `.ai` on the object to get the string that `ap` would print 367 | * `-oi` - Inspect - calls `inspect` on the object 368 | * `-oj` - JSON - calls `to_json` on the object 369 | * `-oJ` - Pretty JSON calls `JSON.pretty_generate` with the object 370 | * `-on` - (default) No Output - output is suppressed 371 | * `-op` - Puts - produces what `puts` would output 372 | * `-os` - To String - calls `to_s` on the object 373 | * `-oy` - YAML - calls `to_yaml` on the object 374 | 375 | All formats will implicitly `require` anything needed to accomplish their task (e.g. `require 'yaml'`). 376 | 377 | The default is `-on` to produce no output at all (unless explicitly coded to do so). If you prefer a different default such as `-op` for _puts_ mode, you can specify that in your `REXE_OPTIONS` environment variable. 378 | 379 | If two letters are provided, the first will be used for tty devices (e.g. the terminal when not redirected or piped), and the second for block devices (e.g. when redirected or piped to another process). 380 | 381 | You may wonder why these formats are provided, given that their functionality could be included in the custom code instead. Here's why: 382 | 383 | * The savings in command line length goes a long way to making these commands more readable and feasible. 384 | * It's much simpler to switch formats, as there is no need to change the code itself. 385 | * This approach enables parameterization of the output format. 386 | 387 | 388 | ### Reading Input from a File 389 | 390 | Rexe also simplifies getting input from a file rather than standard input. The `-f` option takes a filespec and does with its content exactly what it would have done with standard input. This shortens: 391 | 392 | ```bash 393 | $ cat filename.ext | rexe ... 394 | ``` 395 | ...to... 396 | 397 | ```bash 398 | $ rexe -f filename.ext ... 399 | ``` 400 | 401 | This becomes even more useful if you are using files whose extensions are `.yml`, `.yaml`, or `.json` (case insensitively). In this case the input format and mode will be set automatically for you to: 402 | 403 | * `-iy` (YAML) or `-ij` (JSON) depending on the file extension 404 | * `-mb` (one big string mode), which assumes that the most common use case will be to parse the entire file at once 405 | 406 | So the example we gave above: 407 | 408 | ```bash 409 | $ export EUR_RATES_JSON=`curl https://api.exchangeratesapi.io/latest` 410 | $ echo $EUR_RATES_JSON | rexe -mb -ij -oy self 411 | ``` 412 | ...could be changed to: 413 | 414 | ```bash 415 | $ curl https://api.exchangeratesapi.io/latest > eur_rates.json 416 | $ rexe -f eur_rates.json -oy self 417 | ``` 418 | 419 | Another possible win for using `-f` is that since it is a command line option, it could be specified in `REXE_OPTIONS`. This could be useful if you are doing many operations on the same file. 420 | 421 | If you need to override the input mode and format automatically configured for file input, you can simply specify the desired options on the command line _after_ the `-f`: 422 | 423 | ```bash 424 | $ rexe -f eur_rates.json -mb -in 'puts self.class, self[0..20]' 425 | String 426 | {"base":"EUR","rates" 427 | ``` 428 | 429 | 430 | ### 'self' as Default Source Code 431 | 432 | To make rexe even more concise, you do not need to specify any source code when you want that source code to be `self`. This would be the case for simple format conversions, as in JSON to YAML conversion mentioned above: 433 | 434 | ```bash 435 | ➜ ~  rexe -f eur_rates.json -oy 436 | # or 437 | ➜ ~  echo $EUR_RATES_JSON | rexe -mb -ij -oy 438 | ``` 439 | ```yaml 440 | --- 441 | rates: 442 | JPY: 126.63 443 | BRL: 4.3012 444 | NOK: 9.6915 445 | ... 446 | ``` 447 | 448 | This feature is probably only useful in the filter modes, since in the executor mode (`-mn`) self is a new instance of `Object` and hardly ever useful as an output value. 449 | 450 | ### The $RC Global OpenStruct 451 | 452 | For your convenience, the information displayed in verbose mode is available to your code at runtime by accessing the `$RC` global variable, which contains an OpenStruct. Let's print out its contents using YAML: 453 | 454 | ```bash 455 | $ rexe -oy '$RC' 456 | ``` 457 | ```yaml 458 | --- !ruby/object:OpenStruct 459 | table: 460 | :count: 0 461 | :rexe_version: 1.3.1 462 | :start_time: '2019-09-11T13:25:53+07:00' 463 | :source_code: "$RC" 464 | :options: 465 | :input_filespec: 466 | :input_format: :none 467 | :input_mode: :none 468 | :loads: [] 469 | :output_format: :yaml 470 | :output_format_tty: :yaml 471 | :output_format_block: :yaml 472 | :requires: 473 | - yaml 474 | :log_format: :none 475 | :noop: false 476 | modifiable: true 477 | ``` 478 | 479 | Probably most useful in that object at runtime is the record count, accessible with both `$RC.count` and `$RC.i`. This is only really useful in line mode, because in the others it will always be 0 or 1. Here is an example of how you might use it as a kind of progress indicator: 480 | 481 | ```bash 482 | $ find / | rexe -ml -on \ 483 | 'if $RC.i % 1000 == 0; puts %Q{File entry ##{$RC.i} is #{self}}; end' 484 | ``` 485 | ``` 486 | ... 487 | File entry #106000 is /usr/local/Cellar/go/1.11.5/libexec/src/cmd/vendor/github.com/google/pprof/internal/driver/driver_test.go 488 | File entry #107000 is /usr/local/Cellar/go/1.11.5/libexec/src/go/types/testdata/cycles1.src 489 | File entry #108000 is /usr/local/Cellar/go/1.11.5/libexec/src/runtime/os_linux_novdso.go 490 | ... 491 | ``` 492 | 493 | Note that a single quote was used for the Ruby code here; if a double quote were used, the `$RC` would have been interpreted and removed by the shell. 494 | 495 | 496 | ### Implementing Domain Specific Languages (DSL's) 497 | 498 | Defining methods in your loaded files enables you to effectively define a [DSL](https://en.wikipedia.org/wiki/Domain-specific_language) for your command line use. You could use different load files for different projects, domains, or contexts, and define aliases or one line scripts to give them meaningful names. For example, if you had Ansible helper code in `~/projects/ansible-tools/rexe-ansible.rb`, you could define an alias in your startup script: 499 | 500 | ```bash 501 | $ alias rxans="rexe -l ~/projects/ansible-tools/rexe-ansible.rb $*" 502 | ``` 503 | ...and then you would have an Ansible DSL available for me to use by calling `rxans`. 504 | 505 | In addition, since you can also call `pry` on the context of any object, you can provide a DSL in a [REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop) (shell) trivially easily. Just to illustrate, here's how you would open a REPL on the File class: 506 | 507 | ```bash 508 | $ ruby -r pry -e File.pry 509 | # or 510 | $ rexe -r pry File.pry 511 | ``` 512 | 513 | `self` would evaluate to the `File` class, so you could call class methods using only their names: 514 | 515 | ```bash 516 | $ rexe -r pry File.pry 517 | ``` 518 | ``` 519 | [6] pry(File)> size '/etc/passwd' 520 | 6804 521 | [7] pry(File)> directory? '.' 522 | true 523 | [8] pry(File)> file?('/etc/passwd') 524 | true 525 | ``` 526 | 527 | This could be really handy if you call `pry` on a custom object that has methods especially suited to your task: 528 | 529 | ```bash 530 | $ rexe -r wifi-wand,pry WifiWand::MacOsModel.new.pry 531 | ``` 532 | ``` 533 | [1] pry(#)> random_mac_address 534 | "a1:ea:69:d9:ca:05" 535 | [2] pry(#)> connected_network_name 536 | "My WiFi" 537 | ``` 538 | 539 | Ruby is supremely well suited for DSL's since it does not require parentheses for method calls, so calls to your custom methods _look_ like built in language commands and keywords. 540 | 541 | 542 | ### Quotation Marks and Quoting Strings in Your Ruby Code 543 | 544 | One complication of using utilities like rexe where Ruby code is specified on the command line is that you need to be careful about the shell's special treatment of certain characters. For this reason, it is often necessary to quote the Ruby code. You can use single or double quotes to have the shell treat your source code as a single argument. An excellent reference for how they differ is on StackOverflow at https://stackoverflow.com/questions/6697753/difference-between-single-and-double-quotes-in-bash. 545 | 546 | Personally, I find single quotes more useful since I usually don't want special characters in my Ruby code like `$` to be processed by the shell. 547 | 548 | Sometimes it doesn't matter: 549 | 550 | ```bash 551 | $ rexe 'puts "hello"' 552 | hello 553 | $ rexe "puts 'hello'" 554 | hello 555 | ``` 556 | 557 | We can also use `%q` or `%Q`, and sometimes this eliminates the needs for the outer quotes altogether: 558 | 559 | ```bash 560 | $ rexe puts %q{hello} 561 | hello 562 | $ rexe puts %Q{hello} 563 | hello 564 | ``` 565 | 566 | Sometimes the quotes to use on the outside (quoting your command in the shell) need to be chosen based on which quotes are needed on the inside. For example, in the following command, we need double quotes in Ruby in order for interpolation to work, so we use single quotes on the outside: 567 | 568 | ```bash 569 | $ rexe puts '"The time is now #{Time.now}"' 570 | ``` 571 | ``` 572 | The time is now 2019-03-29 16:41:26 +0800 573 | ``` 574 | 575 | In this case we also need to use single quotes on the outside, because we need literal double quotes in a `%Q{}` expression: 576 | 577 | ```bash 578 | $ rexe 'puts %Q{The operating system name is "#{`uname`.chomp}".}' 579 | ``` 580 | ``` 581 | The operating system name is "Darwin". 582 | ``` 583 | 584 | We can eliminate the need for any quotes in the Ruby code using `%Q{}`: 585 | 586 | ```bash 587 | $ rexe puts '%Q{The time is now #{Time.now}}' 588 | ``` 589 | ``` 590 | The time is now 2019-03-29 17:06:13 +0800 591 | ``` 592 | 593 | Of course you can always escape the quotes with backslashes instead, but that is probably more difficult to read. 594 | 595 | 596 | ### No Op Mode 597 | 598 | The `-n` no-op mode will result in the specified source code _not_ being executed. This can sometimes be handy in conjunction with a `-g` (logging) option, if you have are building a rexe command and want to inspect the configuration options before executing the Ruby code. 599 | 600 | 601 | ### Mimicking Method Arguments 602 | 603 | You may want to support arguments in your rexe commands. It's a little kludgy, but you could do this by piping in the arguments as rexe's stdin. 604 | 605 | One of the previous examples downloaded currency conversion rates. To prepare for an example of how to do this, let's find out the available currency codes: 606 | 607 | ```bash 608 | ➜ /  echo $EUR_RATES_JSON | \ 609 | rexe -ij -mb -op "self['rates'].keys.sort.join(' ')" 610 | ``` 611 | ``` 612 | AUD BGN BRL CAD CHF CNY CZK DKK GBP HKD HRK HUF IDR ILS INR ISK JPY KRW MXN MYR NOK NZD PHP PLN RON RUB SEK SGD THB TRY USD ZAR 613 | ``` 614 | 615 | The codes output are the legal arguments that could be sent to rexe's stdin as an argument in the command below. Let's find out the Euro exchange rate for _PHP_, Philippine Pesos: 616 | 617 | ```bash 618 | $ echo PHP | rexe -ml -op -rjson \ 619 | "rate = JSON.parse(ENV['EUR_RATES_JSON'])['rates'][self];\ 620 | %Q{1 EUR = #{rate} #{self}}" 621 | 622 | 1 EUR = 58.986 PHP 623 | ``` 624 | 625 | In this code, `self` is the currency code `PHP` (Philippine Peso). We have accessed the JSON text to parse from the environment variable we previously populated. 626 | 627 | Because we "used up" stdin for the `PHP` argument, we needed to read the JSON data explicitly from the environment variable, and that made the command more complex. A regular Ruby script would handle this more nicely. 628 | 629 | 630 | ### Using the Clipboard for Text Processing 631 | 632 | For editing text in an editor, rexe can be used for text transformations that would otherwise need to be done manually. 633 | 634 | The system's commands for pasting to and copying from the clipboard can handle the moving of the text between the editor and rexe. On the Mac, we have the following commands: 635 | 636 | * `pbcopy` - copies the content of its stdin _to_ the clipboard 637 | * `pbpaste` - copies the content _from_ the clipboard to its stdout 638 | 639 | Let's say we have the following currency codes displayed on the screen (data abridged for brevity): 640 | 641 | ``` 642 | AUD BGN BRL PHP TRY USD ZAR 643 | ``` 644 | 645 | ...and we want to turn them into Ruby symbols for inclusion in Ruby source code as keys in a hash whose values will be the display names of the currencies, e.g "Australian Dollar"). 646 | 647 | We could manually select that text and use system menu commands or keys to copy it to the clipboard, or we could do this: 648 | 649 | ```bash 650 | $ echo AUD BGN BRL PHP TRY USD ZAR | pbcopy 651 | ``` 652 | 653 | After copying this line to the clipboard, we could run this: 654 | 655 | ```bash 656 | $ pbpaste | rexe -ml -op \ 657 | "split.map(&:downcase).map { |s| %Q{ #{s}: '',} }.join(%Q{\n})" 658 | aud: '', 659 | bgn: '', 660 | brl: '', 661 | # ... 662 | ``` 663 | 664 | If I add `| pbcopy` to the rexe command, then that output text would be copied into the clipboard instead of displayed in the terminal, and I could then paste it into my editor. 665 | 666 | Using the clipboard in manual operations is handy, but using it in automated scripts is a very bad idea, since there is only one clipboard per user session. If you use the clipboard in an automated script you risk an error situation if its content is changed by another process, or, conversely, you could mess up another process when you change the content of the clipboard. 667 | 668 | ### Multiline Ruby Commands 669 | 670 | Although rexe is cleanest with short one liners, you may want to use it to include nontrivial Ruby code in your shell script as well. If you do this, you may need to add trailing backslashes to the lines of Ruby code. 671 | 672 | What might not be so obvious is that you will often need to use semicolons as statement separators. For example, here is an example without a semicolon: 673 | 674 | ```bash 675 | $ cowsay hello | rexe -me "print %Q{\u001b[33m} \ 676 | puts to_a" 677 | ``` 678 | ``` 679 | rexe: (eval):1: syntax error, unexpected tIDENTIFIER, expecting '}' 680 | ...new { print %Q{\u001b[33m} puts to_a } 681 | ... ^~~~ 682 | ``` 683 | 684 | The shell combines all backslash terminated lines into a single line of text, so when the Ruby interpreter sees your code, it's all in a single line: 685 | 686 | ```bash 687 | $ cowsay hello | rexe -me "print %Q{\u001b[33m} puts to_a" 688 | ``` 689 | 690 | Adding the semicolon fixes the problem: 691 | 692 | ```bash 693 | $ cowsay hello | rexe -me "print %Q{\u001b[33m}; \ 694 | puts to_a" 695 | ``` 696 | ``` 697 | _______ 698 | < hello > 699 | ------- 700 | \ ^__^ 701 | \ (oo)\_______ 702 | (__)\ )\/\ 703 | ||----w | 704 | || || 705 | ``` 706 | 707 | 708 | ### Clearing the Require and Load Lists 709 | 710 | There may be times when you have specified a load or require on the command line or in the `REXE_OPTIONS` environment variable, but you want to override it for a single invocation. Here are your options: 711 | 712 | 1) Unspecify _all_ the requires or loads with the `-r!` and `-l!` command line options, respectively. 713 | 714 | 2) Unspecify individual requires or loads by preceding the name with `-`, e.g. `-r -rails`. Array subtraction is used, and array subtraction removes _all_ occurrences of each element of the subtracted (subtrahend) array, so: 715 | 716 | ```bash 717 | $ rexe -n -r rails,rails,rails,-rails -gP 718 | ... 719 | :requires=>["pp"], 720 | ... 721 | ``` 722 | 723 | ...would show that the final `-rails` cancelled all the previous `rails` specifications. 724 | 725 | We could have also extracted the requires list programmatically using `$RC` (described above) by doing this: 726 | 727 | ```bash 728 | $ rexe -oP -r rails,rails,rails,-rails '$RC[:options][:requires]' 729 | ["pp"] 730 | ``` 731 | 732 | 733 | ### Clearing _All_ Options 734 | 735 | You can also clear _all_ options specified up to a certain point in time with the _clear options_ option (`-c`). This is especially useful if you have specified options in the `REXE_OPTIONS` environment variable, and want to ignore all of them. 736 | 737 | 738 | ### Comma Separated Requires and Loads 739 | 740 | For consistency with the `ruby` interpreter, rexe supports requires with the `-r` option, but also allows grouping them together using commas: 741 | 742 | ```bash 743 | vvvvvvvvvvvvvvvvvv 744 | $ echo $EUR_RATES_JSON | rexe -r json,awesome_print 'ap JSON.parse(STDIN.read)' 745 | ^^^^^^^^^^^^^^^^^^ 746 | ``` 747 | 748 | Files loaded with the `-l` option are treated the same way. 749 | 750 | 751 | ### Beware of Configured Requires 752 | 753 | Requiring gems and modules for _all_ invocations of rexe will make your commands simpler and more concise, but will be a waste of execution time if they are not needed. You can inspect the execution times to see just how much time is being consumed. For example, we can find out that rails takes about 0.63 seconds to load on one system by observing and comparing the execution times with and without the require (output has been abbreviated using `grep`): 754 | 755 | ```bash 756 | $ rexe -gy -r rails 2>&1 | grep duration 757 | :duration_secs: 0.660138 758 | $ rexe -gy 2>&1 | grep duration 759 | :duration_secs: 0.027781 760 | ``` 761 | (For the above to work, the `rails` gem and its dependencies need to be installed.) 762 | 763 | 764 | ### Operating System Support 765 | 766 | Rexe has been tested successfully on Mac OS, Linux, and Windows Subsystem for Linux (WSL). It is intended as a tool for the Unix shell, and, as such, no attempt is made to support Windows non-Unix shells. 767 | 768 | 769 | ### More Examples 770 | 771 | Here are some more examples to illustrate the use of rexe. 772 | 773 | ---- 774 | 775 | #### Using Rexe as a Simple Calculator 776 | 777 | To output the result to stdout, you can either call `puts` or specify the `-op` option: 778 | 779 | ```bash 780 | $ rexe puts 1 / 3.0 781 | 0.3333333333333333 782 | ``` 783 | 784 | or: 785 | 786 | ```bash 787 | $ rexe -op 1 / 3.0 788 | 0.3333333333333333 789 | ``` 790 | 791 | Since `*` is interpreted by the shell, if we do multiplication, we need to quote the expression: 792 | 793 | ```bash 794 | $ rexe -op '2 * 7' 795 | 14 796 | ``` 797 | 798 | Of course, if you put the `-op` in the `REXE_OPTIONS` environment variable, you don't need to be explicit about the output: 799 | 800 | ```bash 801 | $ export REXE_OPTIONS=-op 802 | $ rexe '2 * 7' 803 | 14 804 | ``` 805 | 806 | ---- 807 | 808 | 809 | #### Outputting ENV 810 | 811 | Output the contents of `ENV` using AmazingPrint [see footnote ^4 regarding ENV.to_s]: 812 | 813 | ```bash 814 | $ rexe -oa ENV 815 | ``` 816 | ``` 817 | { 818 | ... 819 | "LANG" => "en_US.UTF-8", 820 | "PWD" => "/Users/kbennett/work/rexe", 821 | "SHELL" => "/bin/zsh", 822 | ... 823 | } 824 | ``` 825 | 826 | ---- 827 | 828 | #### Reformatting a Command's Output 829 | 830 | Show disk space used/free on a Mac's main hard drive's main partition: 831 | 832 | ```bash 833 | $ df -h | grep disk1s1 | rexe -ml \ 834 | "x = split; puts %Q{#{x[4]} Used: #{x[2]}, Avail: #{x[3]}}" 835 | 91% Used: 412Gi, Avail: 44Gi 836 | ``` 837 | 838 | (Note that `split` is equivalent to `self.split`, and because the `-ml` option is used, `self` is the line of text. 839 | 840 | ---- 841 | 842 | #### Formatting for Numeric Sort 843 | 844 | Show the 3 longest file names of the current directory, with their lengths, in descending order: 845 | 846 | ```bash 847 | $ ls | rexe -ml -op "%Q{[%4d] %s} % [length, self]" | sort -r | head -3 848 | [ 50] Agoda_Booking_ID_9999999 49_–_RECEIPT_enclosed.pdf 849 | [ 40] 679a5c034994544aab4635ecbd50ab73-big.jpg 850 | [ 28] 2018-abc-2019-01-16-2340.zip 851 | ``` 852 | 853 | When you right align numbers using printf formatting, sorting the lines alphabetically will result in sorting them numerically as well. 854 | 855 | ---- 856 | 857 | #### Print yellow (trust me!): 858 | 859 | This uses an [ANSI escape code](https://en.wikipedia.org/wiki/ANSI_escape_code) to output text to the terminal in yellow: 860 | 861 | ```bash 862 | $ cowsay hello | rexe -me "print %Q{\u001b[33m}; puts to_a" 863 | $ # or 864 | $ cowsay hello | rexe -mb "print %Q{\u001b[33m}; puts self" 865 | $ # or 866 | $ cowsay hello | rexe "print %Q{\u001b[33m}; puts STDIN.read" 867 | ``` 868 | ``` 869 | _______ 870 | < hello > 871 | ------- 872 | \ ^__^ 873 | \ (oo)\_______ 874 | (__)\ )\/\ 875 | ||----w | 876 | || ||` 877 | ``` 878 | 879 | 880 | ---- 881 | 882 | #### More YouTube: Differentiating Success and Failure 883 | 884 | Let's take the YouTube example from the "Loading Files" section further. Let's have the video that loads be different for the success or failure of the command. 885 | 886 | If we put this in a load file (such as ~/.rexerc): 887 | 888 | ```ruby 889 | def play(piece_code) 890 | pieces = { 891 | hallelujah: "https://www.youtube.com/watch?v=IUZEtVbJT5c&t=0m20s", 892 | rick_roll: "https://www.youtube.com/watch?v=dQw4w9WgXcQ&t=0m43s", 893 | valkyries: "http://www.youtube.com/watch?v=P73Z6291Pt8&t=0m28s", 894 | wm_tell: "https://www.youtube.com/watch?v=j3T8-aeOrbg", 895 | } 896 | `open #{Shellwords.escape(pieces.fetch(piece_code))}` 897 | end 898 | 899 | 900 | def play_result(success) 901 | play(success ? :hallelujah : :rick_roll) 902 | end 903 | 904 | 905 | # Must pipe the exit code into this Ruby process, 906 | # e.g. using `echo $? | rexe play_result_by_exit_code` 907 | def play_result_by_exit_code 908 | play_result(STDIN.read.chomp == '0') 909 | end 910 | ``` 911 | 912 | Then when we issue a command that succeeds, the Hallelujah Chorus is played [see footnote ^2]: 913 | 914 | ```bash 915 | $ uname; echo $? | rexe play_result_by_exit_code 916 | ``` 917 | 918 | ...but when the command fails, in this case, with an executable which is not found, it plays Rick Astley's "Never Gonna Give You Up": 919 | 920 | ```bash 921 | $ uuuuu; echo $? | rexe play_result_by_exit_code 922 | ``` 923 | 924 | ---- 925 | 926 | #### Reformatting Source Code for Help Text 927 | 928 | Another formatting example...I wanted to reformat this source code... 929 | 930 | ```ruby 931 | 'i' => Inspect 932 | 'j' => JSON 933 | 'J' => Pretty JSON 934 | 'n' => No Output 935 | 'p' => Puts (default) 936 | 's' => to_s 937 | 'y' => YAML 938 | ``` 939 | 940 | ...into something more suitable for my help text. Admittedly, the time it took to do this with rexe probably exceeded the time to do it manually, but it was an interesting exercise and made it easy to try different formats. Here it is, after copying the original text to the clipboard: 941 | 942 | ```bash 943 | $ pbpaste | rexe -ml -op "sub(%q{'}, '-o').sub(%q{' =>}, %q{ })" 944 | -oi Inspect 945 | -oj JSON 946 | -oJ Pretty JSON 947 | -on No Output 948 | -op Puts (default) 949 | -os to_s 950 | -oy YAML 951 | ``` 952 | 953 | 954 | ---- 955 | 956 | #### Currency Conversion 957 | 958 | I travel a lot, and when I visit a country for the first time I often get confused by the exchange rate. I put this in my `~/.rexerc`: 959 | 960 | ```ruby 961 | # Conversion rate to US Dollars 962 | module Curr 963 | module_function 964 | def myr; 4.08 end # Malaysian Ringits 965 | def thb; 31.72 end # Thai Baht 966 | def usd; 1.00 end # US Dollars 967 | def vnd; 23199.50 end # Vietnamese Dong 968 | end 969 | ``` 970 | 971 | If I'm lucky enough to be at my computer when I need to do a conversion, for example, to find the value of 150 Malaysian ringits in US dollars, I can do this: 972 | 973 | ```bash 974 | ➜ rexe git:(master) ✗  rexe puts 150 / Curr.myr 975 | 36.76470588235294 976 | ``` 977 | 978 | Obviously rates will change over time, but this will give me a general idea, which is usually all I need. 979 | 980 | 981 | ---- 982 | 983 | #### Reformatting Grep Output 984 | 985 | I was recently asked to provide a schema for the data in my `rock_books` accounting gem. `rock_books` data is intended to be very small in size, and no data base is used. Instead, the input data is parsed on every run, and reports generated on demand. However, there are data structures (actually class instances) in memory at runtime, and their classes inherit from `Struct`. The definition lines look like this one: 986 | 987 | ```ruby 988 | class JournalEntry < Struct.new(:date, :acct_amounts, :doc_short_name, :description, :receipts) 989 | ``` 990 | 991 | The `grep` command line utility prepends each of these matches with a string like this: 992 | 993 | ``` 994 | lib/rock_books/documents/journal_entry.rb: 995 | ``` 996 | 997 | So this is what worked well for me: 998 | 999 | ```bash 1000 | $ grep Struct **/*.rb | grep -v OpenStruct | rexe -ml -op \ 1001 | "a = \ 1002 | gsub('lib/rock_books/', '') \ 1003 | .gsub('< Struct.new', '') \ 1004 | .gsub('; end', '') \ 1005 | .split('.rb:') \ 1006 | .map(&:strip); \ 1007 | \ 1008 | %q{%-40s %-s} % [a[0] + %q{.rb}, a[1]]" 1009 | ``` 1010 | 1011 | ...which produced this output: 1012 | 1013 | ``` 1014 | cmd_line/command_line_interface.rb class Command (:min_string, :max_string, :action) 1015 | documents/book_set.rb class BookSet (:run_options, :chart_of_accounts, :journals) 1016 | documents/journal.rb class Entry (:date, :amount, :acct_amounts, :description) 1017 | documents/journal_entry.rb class JournalEntry (:date, :acct_amounts, :doc_short_name, :description, :receipts) 1018 | documents/journal_entry_builder.rb class JournalEntryBuilder (:journal_entry_context) 1019 | reports/report_context.rb class ReportContext (:chart_of_accounts, :journals, :page_width) 1020 | types/account.rb class Account (:code, :type, :name) 1021 | types/account_type.rb class AccountType (:symbol, :singular_name, :plural_name) 1022 | types/acct_amount.rb class AcctAmount (:date, :code, :amount, :journal_entry_context) 1023 | types/journal_entry_context.rb class JournalEntryContext (:journal, :linenum, :line) 1024 | ``` 1025 | 1026 | Although there's a lot going on in this code, the vertical and horizontal alignments and spacing make the code straightforward to follow. Here's what it does: 1027 | 1028 | * grep the code base for `"Struct"` 1029 | * exclude references to `"OpenStruct"` with `grep -v` 1030 | * remove unwanted text with `gsub` 1031 | * split the line into 1) a filespec relative to `lib/rockbooks`, and 2) the class definition 1032 | * strip unwanted space because that will mess up the horizontal alignment of the output. 1033 | * use C-style printf formatting to align the text into two columns 1034 | 1035 | ---- 1036 | 1037 | 1038 | ### Conclusion 1039 | 1040 | Rexe is not revolutionary technology, it's just plumbing that removes parsing, formatting, and low level configuration from your command line so that you can focus on the high level task at hand. 1041 | 1042 | When we consider a new piece of software, we usually think "what would this be helpful with now?". However, for me, the power of rexe is not so much what I can do with it in a single use case now, but rather what will I be able to do over time as I accumulate more experience and expertise with it. 1043 | 1044 | I suggest starting to use rexe even for modest improvements in workflow, even if it doesn't seem compelling. There's a good chance that as you use it over time, new ideas will come to you and the workflow improvements will increase exponentially. 1045 | 1046 | A word of caution though -- the complexity and difficulty of _sharing_ your rexe scripts across systems will be proportional to the extent to which you use environment variables and loaded files for configuration and shared code. Be responsible and disciplined in making this configuration and code as clean and organized as possible. 1047 | 1048 | ---- 1049 | 1050 | #### Footnotes 1051 | 1052 | [1]: Rexe is an embellishment of the minimal but excellent `rb` script at https://github.com/thisredone/rb. I started using `rb` and thought of lots of other features I would like to have, so I started working on rexe. 1053 | 1054 | [2]: It's possible that when this page opens in your browser it will not play automatically. You may need to change your default browser, or change the code that opens the URL. Firefox's new (as of March 2019) version 66 suppresses autoplay; you can register exceptions to this policy: open Firefox Preferences, search for "autoplay" and add "https://www.youtube.com". 1055 | 1056 | [3]: Making this truly OS-portable is a lot more complex than it looks on the surface. On Linux, `xdg-open` may not be installed by default. Also, Windows Subsystem for Linux (WSL) out of the box is not able to launch graphical applications. 1057 | 1058 | Here is a _start_ at a method that opens a resource portably across operating systems: 1059 | 1060 | ```ruby 1061 | def open_resource(resource_identifier) 1062 | command = case (`uname`.chomp) 1063 | when 'Darwin' 1064 | 'open' 1065 | when 'Linux' 1066 | 'xdg-open' 1067 | else 1068 | 'start' 1069 | end 1070 | 1071 | `#{command} #{resource_identifier}` 1072 | end 1073 | ``` 1074 | 1075 | [4]: It is an interesting quirk of the Ruby language that `ENV.to_s` returns `"ENV"` and not the contents of the `ENV` object. As a result, many of the other output formats will also return some form of `"ENV"`. You can handle this by specifying `ENV.to_h`. 1076 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require "bundler/gem_tasks" 2 | require "rspec/core/rake_task" 3 | 4 | RSpec::Core::RakeTask.new(:spec) 5 | 6 | task :default => :spec 7 | -------------------------------------------------------------------------------- /bin/console: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | require "bundler/setup" 4 | require "rexe" 5 | 6 | # You can add fixtures and/or initialization code here to make experimenting 7 | # with your gem easier. You can also use a different console, if you like. 8 | 9 | # (If you use this, don't forget to add pry to your Gemfile!) 10 | # require "pry" 11 | # Pry.start 12 | 13 | require "irb" 14 | IRB.start(__FILE__) 15 | -------------------------------------------------------------------------------- /bin/setup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -euo pipefail 3 | IFS=$'\n\t' 4 | set -vx 5 | 6 | bundle install 7 | 8 | # Do any other automated setup that you need to do here 9 | -------------------------------------------------------------------------------- /copy-help-text: -------------------------------------------------------------------------------- 1 | # Here's a sample rexe script that I've been using in my development of rexe. It: 2 | # 3 | # 1) pipes the help output of rexe into another call to rexe, which 4 | # 2) removes the first and last blank lines for easier pasting into the source code, and 5 | # 3) pastes the result back into the clipboard for easy insertion into the editor. 6 | 7 | rexe -h | rexe -mb -op 'self.split("\n")[1..-1]' | pbcopy -------------------------------------------------------------------------------- /exe/rexe: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # 3 | # rexe - Ruby Command Line Executor Filter 4 | # 5 | # Inspired by https://github.com/thisredone/rb 6 | 7 | # frozen_string_literal: true 8 | 9 | 10 | require 'bundler' 11 | require 'date' 12 | require 'optparse' 13 | require 'ostruct' 14 | require 'shellwords' 15 | 16 | class Rexe 17 | 18 | VERSION = '1.7.0' 19 | 20 | PROJECT_URL = 'https://github.com/keithrbennett/rexe' 21 | 22 | 23 | module Helpers 24 | 25 | # Originally the entire run was wrapped in this rescue to make error output more concise. 26 | # However, I am concerned that hiding the stack trace will make it harder to debug, 27 | # and be worse than the ugliness of showing it when it is not needed. So, I am commenting 28 | # out the rescue for now, and will see how it goes. If you have any opinions on this, 29 | # please let me know at https://github.com/keithrbennett/rexe/issues/5 . 30 | # 31 | # (disabled) Try executing code. If error raised, print message (but not stack trace) & exit -1. 32 | def try 33 | yield 34 | # rescue StandardError => e 35 | # $stderr.puts("rexe ERROR: #{e}") 36 | # $stderr.puts('Use the -h option to get help.') 37 | # exit(-1) 38 | end 39 | end 40 | 41 | 42 | class Options < Struct.new( 43 | :input_filespec, 44 | :input_format, 45 | :input_mode, 46 | :loads, 47 | :output_format, 48 | :output_format_tty, 49 | :output_format_block, 50 | :requires, 51 | :log_format, 52 | :noop) 53 | 54 | 55 | def initialize 56 | super 57 | clear 58 | end 59 | 60 | 61 | def clear 62 | self.input_filespec = nil 63 | self.input_format = :none 64 | self.input_mode = :none 65 | self.output_format = :none 66 | self.output_format_tty = :none 67 | self.output_format_block = :none 68 | self.loads = [] 69 | self.requires = [] 70 | self.log_format = :none 71 | self.noop = false 72 | end 73 | end 74 | 75 | 76 | class Lookups 77 | def input_modes 78 | @input_modes ||= { 79 | 'l' => :line, 80 | 'e' => :enumerator, 81 | 'b' => :one_big_string, 82 | 'n' => :none 83 | } 84 | end 85 | 86 | 87 | def input_formats 88 | @input_formats ||= { 89 | 'j' => :json, 90 | 'm' => :marshal, 91 | 'n' => :none, 92 | 'y' => :yaml, 93 | } 94 | end 95 | 96 | 97 | def input_parsers 98 | @input_parsers ||= { 99 | json: ->(string) { JSON.parse(string) }, 100 | marshal: ->(string) { Marshal.load(string) }, 101 | none: ->(string) { string }, 102 | yaml: ->(string) { YAML.load(string) }, 103 | } 104 | end 105 | 106 | 107 | def output_formats 108 | @output_formats ||= { 109 | 'a' => :awesome_print, 110 | 'i' => :inspect, 111 | 'j' => :json, 112 | 'J' => :pretty_json, 113 | 'm' => :marshal, 114 | 'n' => :none, 115 | 'p' => :puts, # default 116 | 'P' => :pretty_print, 117 | 's' => :to_s, 118 | 'y' => :yaml, 119 | } 120 | end 121 | 122 | 123 | def formatters 124 | @formatters ||= { 125 | awesome_print: ->(obj) { obj.ai + "\n" }, 126 | inspect: ->(obj) { obj.inspect + "\n" }, 127 | json: ->(obj) { obj.to_json }, 128 | marshal: ->(obj) { Marshal.dump(obj) }, 129 | none: ->(_obj) { nil }, 130 | pretty_json: ->(obj) { JSON.pretty_generate(obj) }, 131 | pretty_print: ->(obj) { obj.pretty_inspect }, 132 | puts: ->(obj) { require 'stringio'; sio = StringIO.new; sio.puts(obj); sio.string }, 133 | to_s: ->(obj) { obj.to_s + "\n" }, 134 | yaml: ->(obj) { obj.to_yaml }, 135 | } 136 | end 137 | 138 | 139 | def format_requires 140 | @format_requires ||= { 141 | json: 'json', 142 | pretty_json: 'json', 143 | awesome_print: 'awesome_print', 144 | pretty_print: 'pp', 145 | yaml: 'yaml' 146 | } 147 | end 148 | end 149 | 150 | 151 | class CommandLineParser 152 | 153 | include Helpers 154 | 155 | attr_reader :lookups, :options 156 | 157 | def initialize 158 | @lookups = Lookups.new 159 | @options = Options.new 160 | end 161 | 162 | 163 | # Inserts contents of REXE_OPTIONS environment variable at the beginning of ARGV. 164 | private def prepend_environment_options 165 | env_opt_string = ENV['REXE_OPTIONS'] 166 | if env_opt_string 167 | args_to_prepend = Shellwords.shellsplit(env_opt_string) 168 | ARGV.unshift(args_to_prepend).flatten! 169 | end 170 | end 171 | 172 | 173 | private def add_format_requires_to_requires_list 174 | formats = [options.input_format, options.output_format, options.log_format] 175 | requires = formats.map { |format| lookups.format_requires[format] }.uniq.compact 176 | requires.each { |r| options.requires << r } 177 | end 178 | 179 | 180 | private def help_text 181 | unless @help_text 182 | @help_text ||= <<~HEREDOC 183 | 184 | rexe -- Ruby Command Line Executor/Filter -- v#{VERSION} -- #{PROJECT_URL} 185 | 186 | Executes Ruby code on the command line, 187 | optionally automating management of standard input and standard output, 188 | and optionally parsing input and formatting output with YAML, JSON, etc. 189 | 190 | rexe [options] [Ruby source code] 191 | 192 | Options: 193 | 194 | -c --clear_options Clear all previous command line options specified up to now 195 | -f --input_file Use this file instead of stdin for preprocessed input; 196 | if filespec has a YAML and JSON file extension, 197 | sets input format accordingly and sets input mode to -mb 198 | -g --log_format FORMAT Log format, logs to stderr, defaults to -gn (none) 199 | (see -o for format options) 200 | -h, --help Print help and exit 201 | -i, --input_format FORMAT Input format, defaults to -in (None) 202 | -ij JSON 203 | -im Marshal 204 | -in None (default) 205 | -iy YAML 206 | -l, --load RUBY_FILE(S) Ruby file(s) to load, comma separated; 207 | ! to clear all, or precede a name with '-' to remove 208 | -m, --input_mode MODE Input preprocessing mode (determines what `self` will be) 209 | defaults to -mn (none) 210 | -ml line; each line is ingested as a separate string 211 | -me enumerator (each_line on STDIN or File) 212 | -mb big string; all lines combined into one string 213 | -mn none (default); no input preprocessing; 214 | self is an Object.new 215 | -n, --[no-]noop Do not execute the code (useful with -g); 216 | For true: yes, true, y, +; for false: no, false, n 217 | -o, --output_format FORMAT Output format, defaults to -on (no output): 218 | -oa Amazing Print 219 | -oi Inspect 220 | -oj JSON 221 | -oJ Pretty JSON 222 | -om Marshal 223 | -on No Output (default) 224 | -op Puts 225 | -oP Pretty Print 226 | -os to_s 227 | -oy YAML 228 | If 2 letters are provided, 1st is for tty devices, 2nd for block 229 | --project-url Outputs project URL on Github, then exits 230 | -r, --require REQUIRE(S) Gems and built-in libraries to require, comma separated; 231 | ! to clear all, or precede a name with '-' to remove 232 | -v, --version Prints version and exits 233 | 234 | --------------------------------------------------------------------------------------- 235 | 236 | In many cases you will need to enclose your source code in single or double quotes. 237 | 238 | If source code is not specified, it will default to 'self', 239 | which is most likely useful only in a filter mode (-ml, -me, -mb). 240 | 241 | If there is a .rexerc file in your home directory, it will be run as Ruby code 242 | before processing the input. 243 | 244 | If there is a REXE_OPTIONS environment variable, its content will be prepended 245 | to the command line so that you can specify options implicitly 246 | (e.g. `export REXE_OPTIONS="-r awesome_print,yaml"`) 247 | 248 | HEREDOC 249 | 250 | @help_text.freeze 251 | end 252 | 253 | @help_text 254 | end 255 | 256 | 257 | # File file input mode; detects the input mode (JSON, YAML, or None) from the extension. 258 | private def autodetect_file_format(filespec) 259 | extension = File.extname(filespec).downcase 260 | if extension == '.json' 261 | :json 262 | elsif %w{.yml .yaml}.include?(extension) 263 | :yaml 264 | else 265 | :none 266 | end 267 | end 268 | 269 | 270 | private def open_resource(resource_identifier) 271 | command = case (`uname`.chomp) 272 | when 'Darwin' 273 | 'open' 274 | when 'Linux' 275 | 'xdg-open' 276 | else 277 | 'start' 278 | end 279 | 280 | `#{command} #{resource_identifier}` 281 | end 282 | 283 | 284 | # Using 'optparse', parses the command line. 285 | # Settings go into this instance's properties (see Struct declaration). 286 | def parse 287 | 288 | prepend_environment_options 289 | 290 | input_mode_specified = false 291 | input_format_specified = false 292 | output_format_specified = false 293 | 294 | report_unspecified_options = -> do 295 | messages = [] 296 | 297 | unless input_mode_specified 298 | messages << 'Input mode not specified. Defaulting to -mn (none).' 299 | end 300 | unless input_format_specified 301 | messages << 'Input format not specified. Defaulting to -in (none).' 302 | end 303 | unless output_format_specified 304 | messages << 'Output mode not specified. Defaulting to -on (none).' 305 | end 306 | 307 | if messages.any? 308 | $stderr.puts(messages.join("\n")) 309 | $stderr.puts("See help (run with -h) for more information.\n\n") 310 | end 311 | end 312 | 313 | OptionParser.new do |parser| 314 | 315 | parser.on('-c', '--clear_options', 'Clear all previous command line options') do |v| 316 | options.clear 317 | end 318 | 319 | parser.on('-f', '--input_file FILESPEC', 320 | 'Use this file instead of stdin; autodetects YAML and JSON file extensions') do |v| 321 | unless File.exist?(v) 322 | raise "File #{v} does not exist." 323 | end 324 | options.input_filespec = v 325 | options.input_format = autodetect_file_format(v) 326 | input_format_specified = true 327 | if [:json, :yaml].include?(options.input_format) 328 | options.input_mode = :one_big_string 329 | input_mode_specified = true 330 | end 331 | end 332 | 333 | parser.on('-g', '--log_format FORMAT', 'Log format, logs to stderr, defaults to none (see -o for format options)') do |v| 334 | options.log_format = lookups.output_formats[v] 335 | if options.log_format.nil? 336 | raise("Output mode was '#{v}' but must be one of #{lookups.output_formats.keys}.") 337 | end 338 | end 339 | 340 | parser.on("-h", "--help", 'Show help') do |_help_requested| 341 | puts help_text 342 | exit 343 | end 344 | 345 | parser.on('-i', '--input_format FORMAT', 346 | 'Mode with which to parse input values (n = none (default), j = JSON, m = Marshal, y = YAML') do |v| 347 | 348 | options.input_format = lookups.input_formats[v] 349 | if options.input_format.nil? 350 | raise("Input mode was '#{v}' but must be one of #{lookups.input_formats.keys}.") 351 | end 352 | input_format_specified = true 353 | end 354 | 355 | parser.on('-l', '--load RUBY_FILE(S)', 'Ruby file(s) to load, comma separated, or ! to clear') do |v| 356 | if v == '!' 357 | options.loads.clear 358 | else 359 | loadfiles = v.split(',').map(&:strip).map { |s| File.expand_path(s) } 360 | removes, adds = loadfiles.partition { |filespec| filespec[0] == '-' } 361 | 362 | existent, nonexistent = adds.partition { |filespec| File.exist?(filespec) } 363 | if nonexistent.any? 364 | raise("\nDid not find the following files to load: #{nonexistent}\n\n") 365 | else 366 | existent.each { |filespec| options.loads << filespec } 367 | end 368 | 369 | removes.each { |filespec| options.loads -= [filespec[1..-1]] } 370 | end 371 | end 372 | 373 | parser.on('-m', '--input_mode MODE', 374 | 'Mode with which to handle input (-ml, -me, -mb, -mn (default)') do |v| 375 | 376 | options.input_mode = lookups.input_modes[v] 377 | if options.input_mode.nil? 378 | raise("Input mode was '#{v}' but must be one of #{lookups.input_modes.keys}.") 379 | end 380 | input_mode_specified = true 381 | end 382 | 383 | # See https://stackoverflow.com/questions/54576873/ruby-optionparser-short-code-for-boolean-option 384 | # for an excellent explanation of this optparse incantation. 385 | # According to the answer, valid options are: 386 | # -n no, -n yes, -n false, -n true, -n n, -n y, -n +, but not -n -. 387 | parser.on('-n', '--[no-]noop [FLAG]', TrueClass, "Do not execute the code (useful with -g)") do |v| 388 | options.noop = (v.nil? ? true : v) 389 | end 390 | 391 | parser.on('-o', '--output_format FORMAT', 392 | 'Mode with which to format values for output (`-o` + [aijJmnpsy])') do |v| 393 | options.output_format_tty = lookups.output_formats[v[0]] 394 | options.output_format_block = lookups.output_formats[v[-1]] 395 | options.output_format = ($stdout.tty? ? options.output_format_tty : options.output_format_block) 396 | if [options.output_format_tty, options.output_format_block].include?(nil) 397 | raise("Bad output mode '#{v}'; each must be one of #{lookups.output_formats.keys}.") 398 | end 399 | output_format_specified = true 400 | end 401 | 402 | parser.on('-r', '--require REQUIRE(S)', 403 | 'Gems and built-in libraries (e.g. shellwords, yaml) to require, comma separated, or ! to clear') do |v| 404 | if v == '!' 405 | options.requires.clear 406 | else 407 | v.split(',').map(&:strip).each do |r| 408 | if r[0] == '-' 409 | options.requires -= [r[1..-1]] 410 | else 411 | options.requires << r 412 | end 413 | end 414 | end 415 | end 416 | 417 | parser.on('-v', '--version', 'Print version') do 418 | puts VERSION 419 | exit(0) 420 | end 421 | 422 | # Undocumented feature: open Github project with default web browser on a Mac 423 | parser.on('', '--open-project') do 424 | open_resource(PROJECT_URL) 425 | exit(0) 426 | end 427 | 428 | parser.on('', '--project-url') do 429 | puts PROJECT_URL 430 | exit(0) 431 | end 432 | 433 | end.parse! 434 | 435 | # We want to do this after all options have been processed because we don't want any clearing of the 436 | # options (by '-c', etc.) to result in exclusion of these needed requires. 437 | add_format_requires_to_requires_list 438 | 439 | options.requires = options.requires.sort.uniq 440 | options.loads.uniq! 441 | 442 | report_unspecified_options.() if options.log_format == :none 443 | options 444 | 445 | end 446 | end 447 | 448 | 449 | class Main 450 | 451 | include Helpers 452 | 453 | attr_reader :callable, :input_parser, :lookups, 454 | :options, :output_formatter, 455 | :log_formatter, :start_time, :user_source_code 456 | 457 | 458 | def initialize 459 | @lookups = Lookups.new 460 | @start_time = DateTime.now 461 | end 462 | 463 | 464 | private def load_global_config_if_exists 465 | filespec = File.join(Dir.home, '.rexerc') 466 | load(filespec) if File.exist?(filespec) 467 | end 468 | 469 | 470 | private def init_parser_and_formatters 471 | @input_parser = lookups.input_parsers[options.input_format] 472 | @output_formatter = lookups.formatters[options.output_format] 473 | @log_formatter = lookups.formatters[options.log_format] 474 | end 475 | 476 | 477 | # Executes the user specified code in the manner appropriate to the input mode. 478 | # Performs any optionally specified parsing on input and formatting on output. 479 | private def execute(eval_context_object, code) 480 | if options.input_format != :none && options.input_mode != :none 481 | eval_context_object = input_parser.(eval_context_object) 482 | end 483 | 484 | value = eval_context_object.instance_eval(&code) 485 | 486 | unless options.output_format == :none 487 | print output_formatter.(value) 488 | end 489 | rescue Errno::EPIPE 490 | exit(-13) 491 | end 492 | 493 | 494 | # The global $RC (Rexe Context) OpenStruct is available in your user code. 495 | # In order to make it possible to access this object in your loaded files, we are not creating 496 | # it here; instead we add properties to it. This way, you can initialize an OpenStruct yourself 497 | # in your loaded code and it will still work. If you do that, beware, any properties you add will be 498 | # included in the log output. If the to_s of your added objects is large, that might be a pain. 499 | private def init_rexe_context 500 | $RC ||= OpenStruct.new 501 | $RC.count = 0 502 | $RC.rexe_version = VERSION 503 | $RC.start_time = start_time.iso8601 504 | $RC.source_code = user_source_code 505 | $RC.options = options.to_h 506 | 507 | def $RC.i; count end # `i` aliases `count` so you can more concisely get the count in your user code 508 | end 509 | 510 | 511 | private def create_callable 512 | eval("Proc.new { #{user_source_code} }") 513 | end 514 | 515 | 516 | private def lookup_action(mode) 517 | input = options.input_filespec ? File.open(options.input_filespec) : STDIN 518 | { 519 | line: -> { input.each { |l| execute(l.chomp, callable); $RC.count += 1 } }, 520 | enumerator: -> { execute(input.each_line, callable); $RC.count += 1 }, 521 | one_big_string: -> { big_string = input.read; execute(big_string, callable); $RC.count += 1 }, 522 | none: -> { execute(Object.new, callable) } 523 | }.fetch(mode) 524 | end 525 | 526 | 527 | private def output_log_entry 528 | if options.log_format != :none 529 | $RC.duration_secs = Time.now - start_time.to_time 530 | STDERR.puts(log_formatter.($RC.to_h)) 531 | end 532 | end 533 | 534 | 535 | # Bypasses Bundler's restriction on loading gems 536 | # (see https://stackoverflow.com/questions/55144094/bundler-doesnt-permit-using-gems-in-project-home-directory) 537 | private def require!(the_require) 538 | require the_require 539 | rescue LoadError => error 540 | gem_path = `gem which #{the_require}` 541 | if gem_path.chomp.strip.empty? 542 | raise error # re-raise the error, can't fix it 543 | else 544 | load_dir = File.dirname(gem_path) 545 | $LOAD_PATH += load_dir 546 | require the_require 547 | end 548 | end 549 | 550 | 551 | # This class' entry point. 552 | def call 553 | 554 | try do 555 | 556 | @options = CommandLineParser.new.parse 557 | 558 | options.requires.each { |r| require!(r) } 559 | load_global_config_if_exists 560 | options.loads.each { |file| load(file) } 561 | 562 | @user_source_code = ARGV.join(' ') 563 | @user_source_code = 'self' if @user_source_code == '' 564 | 565 | @callable = create_callable 566 | 567 | init_rexe_context 568 | init_parser_and_formatters 569 | 570 | # This is where the user's source code will be executed; the action will in turn call `execute`. 571 | lookup_action(options.input_mode).call unless options.noop 572 | 573 | output_log_entry 574 | end 575 | end 576 | end 577 | end 578 | 579 | 580 | def bundler_run(&block) 581 | # This used to be an unconditional call to with_clean_env but that method is now deprecated: 582 | # [DEPRECATED] `Bundler.with_clean_env` has been deprecated in favor of `Bundler.with_unbundled_env`. 583 | # If you instead want the environment before bundler was originally loaded, 584 | # use `Bundler.with_original_env` 585 | 586 | if Bundler.respond_to?(:with_unbundled_env) 587 | Bundler.with_unbundled_env { block.call } 588 | else 589 | Bundler.with_clean_env { block.call } 590 | end 591 | end 592 | 593 | 594 | bundler_run { Rexe::Main.new.call } 595 | -------------------------------------------------------------------------------- /rexe.gemspec: -------------------------------------------------------------------------------- 1 | 2 | Gem::Specification.new do |spec| 3 | spec.name = "rexe" 4 | 5 | spec.version = -> do 6 | # This is a bit of a kludge. If there is a commented out VERSION line preceding the active line, 7 | # this will read the commented line. 8 | # TODO: Ignore comment lines. 9 | rexe_file = File.join(File.dirname(__FILE__), 'exe', 'rexe') 10 | version_line = File.readlines(rexe_file).grep(/\s*VERSION\s*=\s*'/).first.chomp 11 | version_line.match(/'(.+)'/)[0].gsub("'", '') 12 | end.() 13 | 14 | spec.authors = ["Keith Bennett"] 15 | spec.email = ["keithrbennett@gmail.com"] 16 | 17 | spec.summary = %q{Ruby Command Line Executor} 18 | spec.description = %q{Ruby Command Line Executor} 19 | spec.homepage = "https://github.com/keithrbennett/rexe" 20 | spec.license = "MIT" 21 | 22 | # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host' 23 | # to allow pushing to a single host or delete this section to allow pushing to any host. 24 | if spec.respond_to?(:metadata) 25 | spec.metadata["allowed_push_host"] = "https://rubygems.org/" 26 | 27 | spec.metadata["homepage_uri"] = spec.homepage 28 | spec.metadata["source_code_uri"] = "https://github.com/keithrbennett/rexe" 29 | spec.metadata["changelog_uri"] = "https://github.com/keithrbennett/rexe/blob/master/README.md" 30 | else 31 | raise "RubyGems 2.0 or newer is required to protect against " \ 32 | "public gem pushes." 33 | end 34 | 35 | # Specify which files should be added to the gem when it is released. 36 | # The `git ls-files -z` loads the files in the RubyGem that have been added into git. 37 | spec.files = Dir.chdir(File.expand_path('..', __FILE__)) do 38 | `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) } 39 | end 40 | spec.bindir = "exe" 41 | spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) } 42 | spec.require_paths = ["lib"] 43 | 44 | spec.add_dependency "awesome_print" 45 | 46 | spec.add_development_dependency "bundler", "~> 2.0" 47 | spec.add_development_dependency "os" 48 | spec.add_development_dependency "rake", "~> 12.3" 49 | spec.add_development_dependency "rspec", "~> 3.12" 50 | 51 | # Remove this message (added November 2023) later (maybe July 2024). 52 | spec.post_install_message = <<~MESSAGE 53 | Starting with v1.7.0, awesome_print is now used instead of amazing_print 54 | for fancy human readable output. 55 | MESSAGE 56 | end 57 | -------------------------------------------------------------------------------- /sample-data/eur_rates.json: -------------------------------------------------------------------------------- 1 | { 2 | "base": "EUR", 3 | "rates": { 4 | "BGN": 1.9558, 5 | "NZD": 1.6748, 6 | "ILS": 4.0389, 7 | "RUB": 72.6133, 8 | "CAD": 1.5082, 9 | "USD": 1.1321, 10 | "PHP": 58.553, 11 | "CHF": 1.1326, 12 | "ZAR": 15.7631, 13 | "AUD": 1.5771, 14 | "JPY": 126.76, 15 | "TRY": 6.535, 16 | "HKD": 8.8788, 17 | "MYR": 4.658, 18 | "THB": 35.955, 19 | "HRK": 7.435, 20 | "NOK": 9.602, 21 | "IDR": 15954.12, 22 | "DKK": 7.4643, 23 | "CZK": 25.623, 24 | "HUF": 321.9, 25 | "GBP": 0.8629, 26 | "MXN": 21.236, 27 | "KRW": 1283.0, 28 | "ISK": 135.2, 29 | "SGD": 1.5318, 30 | "BRL": 4.3884, 31 | "PLN": 4.2796, 32 | "INR": 78.2915, 33 | "RON": 4.7598, 34 | "CNY": 7.5939, 35 | "SEK": 10.4788 36 | }, 37 | "date": "2019-04-12" 38 | } -------------------------------------------------------------------------------- /spec/dummy.rb: -------------------------------------------------------------------------------- 1 | # This file and class is used for testing the loading of Ruby files. 2 | # Tests will refer to the Dummy class and see if that generates an error. 3 | class Dummy 4 | end -------------------------------------------------------------------------------- /spec/rexe_spec.rb: -------------------------------------------------------------------------------- 1 | require_relative 'spec_helper' 2 | require 'json' 3 | require 'os' 4 | require 'yaml' 5 | 6 | 7 | # It would be nice to test the behavior of requires, loads, and the global config file, 8 | # but those are difficult because they involve modifying directories of the development 9 | # machine that they should not modify. I'm thinking about making the rules configurable, 10 | # but that makes the executable more complex, probably without necessity. 11 | 12 | 13 | RSpec.describe 'Rexe integration tests' do 14 | 15 | let(:test_data) { [ { 'color' => 'blue' }, 200 ] } 16 | let(:test_data_string) { %Q{[ { 'color' => 'blue' }, 200 ]} } 17 | let(:load_filespec) { File.join(File.dirname(__FILE__), 'dummy.rb') } 18 | let(:readme_filespec) { File.join(File.dirname(__FILE__), '..', 'README.md') } 19 | let(:readme_text) { File.read(readme_filespec) } 20 | let(:readme_lines) { File.readlines(readme_filespec) } 21 | 22 | context '-v --version option' do 23 | specify 'version returned with --version is a valid version string' do 24 | 25 | expect(RUN.("#{REXE_FILE} --version")).to match(/\d+\.\d+\.\d+/) 26 | end 27 | 28 | specify 'version returned with -v is a valid version string' do 29 | expect(RUN.("#{REXE_FILE} -v")).to match(/\d+\.\d+\.\d+/) 30 | end 31 | 32 | specify 'version in README help output matches current version' do 33 | 34 | software_version = RUN.("#{REXE_FILE} --version").chomp 35 | version_line_regex = %r{rexe -- Ruby Command Line Executor/Filter -- v} 36 | 37 | lines_to_inspect = readme_lines.grep(version_line_regex) 38 | expect(lines_to_inspect.size).to eq(1) 39 | 40 | readme_version_line = lines_to_inspect.first 41 | readme_version = readme_version_line.split(' -- ')[2][1..-1] # remove 'v' 42 | 43 | unless software_version == readme_version 44 | fail "Version in software was #{software_version.inspect} but " + 45 | "version in README help was #{readme_version.inspect}." 46 | end 47 | end 48 | end 49 | 50 | 51 | context '-h help text' do 52 | specify 'includes version' do 53 | expect(RUN.("#{REXE_FILE} 2>/dev/null -h")).to include(`#{REXE_FILE} --version`.chomp) 54 | end 55 | 56 | specify 'includes Github URL' do 57 | expect(RUN.("#{REXE_FILE} -h")).to include('https://github.com/keithrbennett/rexe') 58 | end 59 | end 60 | 61 | 62 | context '-m input modes' do # (not formats) 63 | 64 | context '-mb big string mode' do 65 | specify 'all input is considered a single string object' do 66 | expect(RUN.(%Q{echo "ab\ncd" | #{REXE_FILE} -c -mb -op reverse})).to eq("\ndc\nba\n") 67 | end 68 | 69 | specify 'record count does not exceed 0' do 70 | expect(RUN.(%Q{echo "a\nb\nc" | #{REXE_FILE} -c -mb -op '$RC.i'})).to eq("0\n") 71 | end 72 | end 73 | 74 | 75 | context '-ml line mode' do 76 | specify 'each line is processed separately' do 77 | expect(RUN.(%Q{echo "ab\ncd" | #{REXE_FILE} -c -ml -op reverse})).to eq("ba\ndc\n") 78 | end 79 | 80 | specify 'object count works in numbers > 1' do 81 | expect(RUN.(%Q{echo "a\nb\nc" | #{REXE_FILE} -c -ml -op '$RC.i'})).to eq("0\n1\n2\n") 82 | end 83 | end 84 | 85 | 86 | context '-me enumerator mode' do 87 | specify 'self is an Enumerator' do 88 | expect(RUN.(%Q{echo "ab\ncd" | #{REXE_FILE} -c -me -op self.class.to_s}).chomp).to eq('Enumerator') 89 | end 90 | 91 | specify 'record count does not exceed 0' do 92 | expect(RUN.(%Q{echo "a\nb\nc" | #{REXE_FILE} -c -me -op '$RC.i'})).to eq("0\n") 93 | end 94 | end 95 | 96 | 97 | context '-mn no input mode' do 98 | specify 'in no input mode (-mn), code is executed without input' do 99 | expect(RUN.(%Q{#{REXE_FILE} -c -mn -op '64.to_s(8)'})).to start_with('100') 100 | end 101 | 102 | specify '-mn option outputs last evaluated value' do 103 | expect(RUN.(%Q{#{REXE_FILE} -c -mn -op 42}).chomp).to eq('42') 104 | end 105 | 106 | specify 'record count does not exceed 0' do 107 | expect(RUN.(%Q{echo "a\nb\nc" | #{REXE_FILE} -c -mn -op '$RC.i'})).to eq("0\n") 108 | end 109 | end 110 | end 111 | 112 | 113 | context '-o output formats' do 114 | 115 | specify '-on no output format results in no output' do 116 | expect(RUN.(%Q{#{REXE_FILE} -c -mn -on 42}).chomp).to eq('') 117 | end 118 | 119 | specify 'output format defaults to -on (no output)' do 120 | expect(RUN.(%Q{#{REXE_FILE} -c -mn 42}).chomp).to eq('') 121 | end 122 | 123 | specify '-oj JSON output formatting is correct' do 124 | command = %Q{#{REXE_FILE} -c -mn -oj "[ { 10 => 100 }, 200 ]" } 125 | expect(RUN.(command).chomp).to eq(%q{[{"10":100},200]}) 126 | end 127 | 128 | specify '-oJ Pretty JSON output formatting is correct' do 129 | actual_lines_stripped = RUN.(%Q{#{REXE_FILE} -c -mn -oJ '[ { 10 => 100 }, 200 ]' }).split("\n").map(&:strip) 130 | expected_lines_stripped = ['[', '{', '"10": 100', '},','200', ']' ] 131 | expect(actual_lines_stripped).to eq(expected_lines_stripped) 132 | end 133 | 134 | specify '-oy YAML output formatting is correct' do 135 | expect(RUN.(%Q{#{REXE_FILE} -c -mn -oy '[ { 10 => 100 }, 200 ]' }).chomp).to eq( \ 136 | "--- 137 | - 10: 100 138 | - 200") 139 | end 140 | 141 | specify 'inspect (-oi), and to_s (-os) mode return equal and correct strings' do 142 | inspect_output = RUN.(%Q{#{REXE_FILE} -c -mn -oi '[ { 10 => 100 }, 200 ]' }) 143 | to_s_output = RUN.(%Q{#{REXE_FILE} -c -mn -os '[ { 10 => 100 }, 200 ]' }) 144 | 145 | expect(to_s_output).to eq(inspect_output) 146 | expect(to_s_output).to eq(%Q{[{10=>100}, 200]\n}) 147 | end 148 | 149 | 150 | specify '-op (puts) format works correctly' do 151 | puts_output = RUN.(%Q{#{REXE_FILE} -c -mn -op "[ { 10 => 100 }, 200 ]"}) 152 | expect(puts_output).to eq( %Q{{10=>100}\n200\n}) 153 | end 154 | 155 | specify '-om marshall mode works' do 156 | data = RUN.(%Q{#{REXE_FILE} -c -mn -om "[ { 10 => 100 }, 200 ]"}) 157 | reconstructed_array = Marshal.load(data) 158 | expect(reconstructed_array).to be_a(Array) 159 | expect(reconstructed_array).to eq([ { 10 => 100 }, 200] ) 160 | end 161 | end 162 | 163 | 164 | context '-g logging' do 165 | 166 | specify '-gy option enables log in YAML format mode' do 167 | text = RUN.(%Q{#{REXE_FILE} -c -in -mn -gy -on String.new 2>&1}) 168 | reconstructed_hash = YAML.load(text) 169 | expect(reconstructed_hash).to be_a(Hash) 170 | expect(reconstructed_hash[:count]).to eq(0) 171 | expect(reconstructed_hash.keys).to include(:duration_secs) 172 | expect(reconstructed_hash.keys).to include(:options) 173 | expect(reconstructed_hash.keys).to include(:rexe_version) 174 | expect(reconstructed_hash[:source_code]).to eq('String.new') 175 | expect(reconstructed_hash.keys).to include(:start_time) 176 | end 177 | 178 | specify '-gJ option enables log in Pretty JSON format mode' do 179 | text = RUN.(%Q{#{REXE_FILE} -c -mn -gJ -on String.new 2>&1}) 180 | expect(text.count("\n") > 3).to eq(true) 181 | reconstructed_hash = JSON.parse(text) 182 | 183 | expect(reconstructed_hash).to be_a(Hash) 184 | expect(reconstructed_hash.keys).to include('duration_secs') 185 | expect(reconstructed_hash.keys).to include('options') 186 | expect(reconstructed_hash.keys).to include('rexe_version') 187 | expect(reconstructed_hash.keys).to include('start_time') 188 | 189 | # Note below that the keys below are parsed as a String, not its original type, Symbol: 190 | expect(reconstructed_hash['count']).to eq(0) 191 | expect(reconstructed_hash['source_code']).to eq('String.new') 192 | end 193 | 194 | 195 | specify '-gj option enables log in standard JSON format mode' do 196 | text = RUN.(%Q{#{REXE_FILE} -c -mn -gj -on String.new 2>&1}) 197 | expect(text.count("\n") == 1).to eq(true) 198 | reconstructed_hash = JSON.parse(text) 199 | 200 | expect(reconstructed_hash).to be_a(Hash) 201 | expect(reconstructed_hash.keys).to include('duration_secs') 202 | expect(reconstructed_hash.keys).to include('options') 203 | expect(reconstructed_hash.keys).to include('rexe_version') 204 | expect(reconstructed_hash.keys).to include('start_time') 205 | 206 | # Note below that the keys below are parsed as a String, not its original type, Symbol: 207 | expect(reconstructed_hash['count']).to eq(0) 208 | expect(reconstructed_hash['source_code']).to eq('String.new') 209 | end 210 | 211 | 212 | specify '-ga option enables log in Awesome Print format mode' do 213 | text = RUN.(%Q{#{REXE_FILE} -c -mn -ga -on String.new 2>&1}) 214 | expect(text).to include(':count =>') 215 | expect(text).to include(':rexe_version =>') 216 | end 217 | 218 | specify '-gn option disables log' do 219 | expect(RUN.(%Q{#{REXE_FILE} -c -in -gn -mn -on 3 2>&1}).chomp).to eq('') 220 | end 221 | 222 | specify 'not specifying a -g option disables log' do 223 | expect(RUN.(%Q{#{REXE_FILE} -c -in -mn -on 3 2>&1}).chomp).to eq('') 224 | end 225 | 226 | specify '-gm marshall mode works' do 227 | data = RUN.(%Q{#{REXE_FILE} -c -mn -on -gm String.new 2>&1}) 228 | expect(data.count("\x00") > 0).to eq(true) 229 | reconstructed_hash = Marshal.load(data) 230 | expect(reconstructed_hash).to be_a(Hash) 231 | expect(reconstructed_hash[:count]).to eq(0) 232 | expect(reconstructed_hash.keys).to include(:duration_secs) 233 | expect(reconstructed_hash.keys).to include(:options) 234 | expect(reconstructed_hash.keys).to include(:rexe_version) 235 | expect(reconstructed_hash[:source_code]).to eq('String.new') 236 | expect(reconstructed_hash.keys).to include(:start_time) 237 | end 238 | 239 | specify 'Puts (-gp), inspect (-gi), and to_s (-gs) mode return similar strings' do 240 | puts_output = RUN.(%Q{#{REXE_FILE} -c -mn -on -gp String.new 2>&1}) 241 | inspect_output = RUN.(%Q{#{REXE_FILE} -c -mn -on -gi String.new 2>&1}) 242 | to_s_output = RUN.(%Q{#{REXE_FILE} -c -mn -on -gs String.new 2>&1}) 243 | 244 | outputs = [puts_output, inspect_output, to_s_output] 245 | 246 | outputs.each do |output| 247 | expect(output).to match(/^{:/) 248 | expect(output).to match(/}$/) 249 | expect(output).to include(':count=>0') 250 | expect(output).to include(':rexe_version=>') 251 | expect(output.count("\n")).to eq(1) 252 | end 253 | end 254 | end 255 | 256 | 257 | context '-r requires' do 258 | specify 'requiring using -r works' do 259 | RUN.("#{REXE_FILE} -c -mn -op -r yaml YAML") # just refer to the YAML module and see if it breaks 260 | expect($?.exitstatus).to eq(0) 261 | end 262 | 263 | specify 'clearing requires using -r ! works' do 264 | command = "#{REXE_FILE} 2>/dev/null -c -mn -op -r yaml -r! YAML" 265 | RUN.(command) # just refer to the YAML module and see if it breaks 266 | expect($?.exitstatus).not_to eq(0) 267 | end 268 | 269 | specify 'clearing a single require using -r -gem works' do 270 | command = "#{REXE_FILE} 2>/dev/null -c -mn -op -r yaml -r -yaml YAML" 271 | RUN.(command) # just refer to the YAML module and see if it breaks 272 | expect($?.exitstatus).not_to eq(0) 273 | end 274 | end 275 | 276 | context '-l loads' do 277 | 278 | specify 'loading using -l works' do 279 | RUN.("#{REXE_FILE} -c -mn -op -l #{load_filespec} Dummy") # just refer to the YAML module and see if it breaks 280 | expect($?.exitstatus).to eq(0) 281 | end 282 | 283 | specify 'clearing loads using -l ! works' do 284 | command = "#{REXE_FILE} 2>/dev/null -c -mn -op -l #{load_filespec} -l! Dummy" 285 | RUN.(command) # just refer to the YAML module and see if it breaks 286 | expect($?.exitstatus).not_to eq(0) 287 | end 288 | 289 | specify 'clearing a single load using -r -file works' do 290 | command = "#{REXE_FILE} 2>/dev/null -c -mn -op -l #{load_filespec} -l -#{load_filespec} Dummy" 291 | RUN.(command) # just refer to the YAML module and see if it breaks 292 | expect($?.exitstatus).not_to eq(0) 293 | end 294 | 295 | let (:fs1) { 'spec/dummy.rb'} 296 | let (:fs2) { 'spec/../spec/dummy.rb'} 297 | 298 | specify 'two different load filespecs that point to the same absolute location are treated as one' do 299 | command = "#{REXE_FILE} -c -n -gy -l #{fs1} -l #{fs2} 2>&1" 300 | yaml = RUN.(command) # just refer to the YAML module and see if it breaks 301 | config = YAML.load(yaml) 302 | expect(config[:options][:loads].size).to eq(1) 303 | end 304 | end 305 | 306 | 307 | context '$RC.i rexe context record count' do 308 | specify 'the record count is available as $RC.count' do 309 | expect(RUN.(%Q{echo "a\nb\nc" | rexe -ml -op 'self + $RC.count.to_s'})).to eq("a0\nb1\nc2\n") 310 | end 311 | 312 | specify 'the record count is available as $RC.i' do 313 | expect(RUN.(%Q{echo "a\nb\nc" | rexe -ml -op 'self + $RC.i.to_s'})).to eq("a0\nb1\nc2\n") 314 | end 315 | end 316 | 317 | 318 | context '-f file input' do 319 | specify '-f: text file is read correctly' do 320 | text = "1\n2\n3\n" 321 | file_containing(text) do |filespec| 322 | expect(RUN.(%Q{rexe -f #{filespec} -mb -op self})).to eq(text) 323 | end 324 | end 325 | 326 | specify 'text file options are set correctly (not overrided)' do 327 | file_containing('', '.txt') do |filespec| 328 | log_yaml = RUN.(%Q{rexe -f #{filespec} -n -gy 2>&1}) 329 | log = YAML.load(log_yaml) 330 | expect(log[:options][:input_mode]).to eq(:none) 331 | expect(log[:options][:input_format]).to eq(:none) 332 | end 333 | end 334 | 335 | specify 'YAML file is parsed as YAML without specifying -mb or -iy' do 336 | array = [1,4,7] 337 | text = array.to_yaml 338 | %w(.yml .yaml .yaML).each do |extension| 339 | file_containing(text, extension) do |filespec| 340 | expect(RUN.(%Q{rexe -f #{filespec} -op 'self == [1,4,7]' }).chomp).to eq('true') 341 | log_yaml = RUN.(%Q{rexe -f #{filespec} -n -on -gy nil 2>&1 }) 342 | log = YAML.load(log_yaml) 343 | expect(log[:options][:input_mode]).to eq(:one_big_string) 344 | expect(log[:options][:input_format]).to eq(:yaml) 345 | end 346 | end 347 | end 348 | 349 | specify 'JSON file is parsed as JSON without specifying -mb or -ij' do 350 | array = [1,4,7] 351 | text = array.to_json 352 | %w(.json .JsOn).each do |extension| 353 | file_containing(text, extension) do |filespec| 354 | expect(RUN.(%Q{rexe -f #{filespec} -op 'self == [1,4,7]' }).chomp).to eq('true') 355 | log_yaml = RUN.(%Q{rexe -f #{filespec} -n -on -gy nil 2>&1 }) 356 | log = YAML.load(log_yaml) 357 | expect(log[:options][:input_mode]).to eq(:one_big_string) 358 | expect(log[:options][:input_format]).to eq(:json) 359 | end 360 | end 361 | end 362 | end 363 | 364 | context '-n no op' do 365 | specify '-n suppresses evaluation' do 366 | expect(RUN.(%Q{rexe -op '"hello"'})).to eq("hello\n") 367 | expect(RUN.(%Q{rexe -op -n '"hello"'})).to eq('') 368 | end 369 | end 370 | 371 | context 'source code' do 372 | specify 'source code is "self" when there no source code is specified' do 373 | expect(RUN.(%Q{echo '[1,2]' | rexe -ij -ml -oy})).to eq("---\n- 1\n- 2\n") 374 | end 375 | end 376 | 377 | context 'article text metadata' do 378 | specify ' should not be copied to the readme' do 379 | expect(readme_text).not_to include("---\ntitle: ") 380 | expect(readme_text).not_to include("[Caution: This is a long article!") 381 | end 382 | end 383 | 384 | context 'important strings are frozen' do 385 | [ 386 | 'Rexe::VERSION', 387 | 'Rexe::PROJECT_URL', 388 | 'Rexe::CommandLineParser.new.send(:help_text)' 389 | ].each do |important_string| 390 | it "prevents modifying '#{important_string}' because it is frozen" do 391 | command = %Q{rexe -op '#{important_string} << "foo"' 2>&1} 392 | output = `#{command}` 393 | expect(output).to match(/can't modify frozen String/) 394 | end 395 | end 396 | end 397 | end 398 | -------------------------------------------------------------------------------- /spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | require "bundler/setup" 2 | 3 | REXE_FILE = File.expand_path(File.join(File.dirname(__FILE__), '..', 'exe', 'rexe')) 4 | 5 | # Use this so that testing rexe with requires not in the bundle will load successfully: 6 | RUN = ->(command) do 7 | # This used to be an unconditional call to with_clean_env but that method is now deprecated: 8 | # [DEPRECATED] `Bundler.with_clean_env` has been deprecated in favor of `Bundler.with_unbundled_env`. 9 | # If you instead want the environment before bundler was originally loaded, 10 | # use `Bundler.with_original_env` 11 | if Bundler.respond_to?(:with_unbundled_env) 12 | Bundler.with_unbundled_env { `#{command}` } 13 | else 14 | Bundler.with_clean_env { `#{command}` } 15 | end 16 | end 17 | 18 | RSpec.configure do |config| 19 | # Enable flags like --only-failures and --next-failure 20 | config.example_status_persistence_file_path = ".rspec_status" 21 | 22 | # Disable RSpec exposing methods globally on `Module` and `main` 23 | config.disable_monkey_patching! 24 | 25 | config.expect_with :rspec do |c| 26 | c.syntax = :expect 27 | end 28 | end 29 | 30 | 31 | # From the trick_bag gem: 32 | require 'tempfile' 33 | 34 | # For the easy creation and deletion of a temp file populated with text, 35 | # wrapped around the code block you provide. 36 | # 37 | # @param text the text to write to the temporary file 38 | # @yield filespec of the temporary file 39 | def file_containing(text, extension = nil) 40 | raise "This method must be called with a code block." unless block_given? 41 | 42 | filespec = nil 43 | begin 44 | Tempfile.open(['rexe-spec-', extension]) do |file| 45 | file << text 46 | filespec = file.path 47 | end 48 | yield(filespec) 49 | ensure 50 | File.delete(filespec) if filespec && File.exist?(filespec) 51 | end 52 | end 53 | --------------------------------------------------------------------------------