└── README.rst /README.rst: -------------------------------------------------------------------------------- 1 | xonsh Quickstart 2 | ================ 3 | xonsh_ is an interactive system shell that is a Python superset. It is 4 | very handy. I'm not going to give you the commercial for it here. If you 5 | know Python and you use Bash ever, you already know why you want more 6 | Python in your shell. 7 | 8 | This guide exists because, while the xonsh documentation is well-written 9 | and complete, it involves a lot of reading to be productive with xonsh. 10 | The goal of this guide is to allow you to become productive as quickly 11 | as possible in the interactive prompt, which is accomplished by omitting 12 | a lot of details. Refer to the official documentation for more 13 | information if you need it. 14 | 15 | I have another tutorial on `administrative scripting with Python`_ which 16 | is complementary to this one in some ways, in that it shows how to do a 17 | lot of common shell tasks in native Python. 18 | 19 | This page will not teach any Python. If you want to use Xonsh and don't 20 | know Python, you're going to need that. Try the `official tutorial`_. I 21 | may at some point write a Python quickstart guide, but I wouldn't hold 22 | your breath. This guide also assumes some familiarity with Bash or other 23 | POSIX shell. 24 | 25 | .. contents:: 26 | 27 | .. _xonsh: https://xon.sh/ 28 | 29 | .. _administrative scripting with Python: 30 | https://github.com/ninjaaron/replacing-bash-scripting-with-python 31 | 32 | .. _official tutorial: https://docs.python.org/3/tutorial/index.html 33 | 34 | A Tale of Two Modes 35 | ------------------- 36 | Shell syntax is optimized for launching processes. Python is not. The 37 | xonsh solution is to have two modes. 38 | 39 | - xonsh has Python mode and subprocess mode which have different 40 | syntax. 41 | - Switching between modes is usually implicit, but can be forced_. 42 | - Python mode is just Python with a few extras. 43 | - Subprocess mode looks a bit like Bash [#]_, but it also has big 44 | differences. 45 | - xonsh provides constructs which work across both modes. Understanding 46 | these constructs is key to effective use of the shell. 47 | 48 | Subprocess mode requires a little description, which you will find in 49 | the next section. Following that, there is a description of the 50 | mechanisms one can use to move data back and forth between these two 51 | modes, and finally some configuration tips. 52 | 53 | .. _forced: Substitutions_ 54 | .. [#] I say "Bash" frequently in this guide, but I am really referring 55 | to POSIX shells in general. 56 | 57 | Subprocess Mode 58 | --------------- 59 | Similarities to POSIX 60 | ~~~~~~~~~~~~~~~~~~~~~ 61 | Subprocess mode is automatic when a line begins with a name that doesn't 62 | exist in the current scope. In this mode Syntax is superficially similar 63 | to any POSIX shell: 64 | 65 | - A list of whitespace-separated arguments is handed to the executable 66 | as strings. 67 | - these arguments can be quoted to escape special characters and 68 | whitespace. 69 | - Basic globbing also works, and ``**`` may be used in Python 3.5+ for 70 | recursive globbing as in some other popular shells. 71 | - ``&&`` and ``||`` work as in a POSIX shell, but the creators encourage 72 | the use of ``and`` and ``or`` instead. 73 | - Backgrounding processes with ``&`` works. See `job control`_ for more. 74 | - Pipes and I/O redirection also works in a similar way to the shell. 75 | ``| > >> < 2>``, etc. 76 | 77 | Heredocs are not included. xonsh provides additional, more explicit 78 | `redirection syntax`_, but the standard POSIX forms work just fine. I'm 79 | not sure to what extent more advanced use of file descriptors is 80 | supported, but ``2>&1`` works (I believe it may be special-cased). 81 | 82 | .. code:: sh 83 | 84 | $ ls -l 85 | total 40 86 | -rw-r--r-- 1 ninjaaron ninjaaron 26872 Oct 3 23:01 out.html 87 | -rw-r--r-- 1 ninjaaron ninjaaron 11313 Oct 4 21:32 README.rst 88 | $ touch 'filename with spaces' 89 | $ ls -l 'filename with spaces' 90 | -rw-r--r-- 1 ninjaaron ninjaaron 0 Oct 3 21:15 'filename with spaces' 91 | $ # here's a glob 92 | $ echo /usr/* 93 | /usr/bin /usr/include /usr/lib /usr/lib32 /usr/lib64 /usr/local /usr/sbin /usr/share /usr/src 94 | $ sudo apt-get update && sudo apt-get dist-upgrade 95 | [...] 96 | $ # alternative: sudo apt-get update and sudo apt-get dist-upgrade 97 | $ firefox & 98 | $ # firefox is running along on its merry way. 99 | $ echo /usr/l* | tr a-z A-Z 100 | /USR/LIB /USR/LIB32 /USR/LIB64 /USR/LOCAL 101 | 102 | Differences from POSIX 103 | ~~~~~~~~~~~~~~~~~~~~~~ 104 | - Strings are Python 3 strings in every way. 105 | - Backslash escapes in arguments are not allowed. 106 | - Strings literals are the only way to escape special shell characters. 107 | (... excluding so-called `subprocess macros`_...) 108 | 109 | .. code:: sh 110 | 111 | $ rm filename\ with\ spaces 112 | /usr/bin/rm: cannot remove 'filename\': No such file or directory 113 | /usr/bin/rm: cannot remove 'with\': No such file or directory 114 | /usr/bin/rm: cannot remove 'spaces': No such file or directory 115 | $ rm 'filename with spaces' 116 | $ 117 | 118 | - No brace expansion yet_ (iterables can be expanded. see: `Python 119 | Substitution`_) 120 | - quoting part of a string with special characters and leaving another 121 | part unquoted (perhaps for the use of a glob character or brace 122 | expansion) is not permitted. The creators of xonsh find this behavior 123 | to be "insane_". 124 | 125 | .. code:: sh 126 | 127 | $ touch "filename with spaces" 128 | $ ls -l "filename with"* 129 | /usr/bin/ls: cannot access '"filename with"*': No such file or directory 130 | $ # ^ someone else's idea of sanity. 131 | $ # xonsh has additional globbing mechanisms to compensate for this 132 | $ # lack, which are covered in the next section. 133 | 134 | - Command substitution in subprocess mode only works with ``$()``. 135 | Backticks mean something else in xonsh. Both of these features will be 136 | covered in more detail in the following section. 137 | 138 | That about covers it for the quickstart to subprocesses mode. The next 139 | section deals with passing data between the two modes. 140 | 141 | .. _redirection syntax: 142 | https://xon.sh/tutorial.html#input-output-redirection 143 | 144 | .. _subprocess macros: 145 | https://xon.sh/tutorial_macros.html#subprocess-macros 146 | 147 | .. _yet: 148 | https://github.com/xonsh/xonsh/pull/2868 149 | 150 | .. _insane: 151 | https://xon.sh/tutorial_subproc_strings.html?highlight=insane#the-quotes-stay 152 | 153 | .. _job control: 154 | https://xon.sh/tutorial.html#job-control 155 | 156 | Going between the Modes 157 | ----------------------- 158 | There are several special xonsh constructs that work both in subprocess 159 | mode and in Python mode which can be useful for carting data around, 160 | though the first feature we'll cover will be globbing, which isn't 161 | exactly a way to move data between the modes. 162 | 163 | Globs 164 | ~~~~~ 165 | Aside from the unquoted globbing behavior in subprocess mode, xonsh 166 | supports `regex globbing`_ everywhere with backticks. This feels overkill 167 | most of the time, but is extremely useful when you need it. It is also 168 | somewhat necessitated by the omission of brace expansion. 169 | 170 | .. code:: sh 171 | 172 | $ echo `/usr/l.*` 173 | /usr/lib /usr/lib32 /usr/lib64 /usr/local 174 | $ # in a folder containing folders with dates as names... 175 | $ ls -d `18\.0[5-6].*` 176 | 18.05.13 18.05.20 18.06.03 18.06.22 18.06.24 177 | 18.05.19 18.05.27 18.06.17 18.06.23 178 | $ # in Bash this would be `ls -d 18.0{5..6}*` 179 | 180 | Likewise, xonsh supports normal globbing syntax everywhere through the 181 | use of g-strings. These are created with backticks and a ``g`` prefix. 182 | 183 | .. code:: shell 184 | 185 | $ ls -ld g`/usr/l*` 186 | drwxr-xr-x 1 root root 137232 Sep 24 20:50 /usr/lib 187 | drwxr-xr-x 1 root root 38424 Sep 24 20:38 /usr/lib32 188 | lrwxrwxrwx 1 root root 3 Aug 21 16:21 /usr/lib64 -> lib 189 | drwxr-xr-x 1 root root 72 Mar 26 2017 /usr/local 190 | 191 | This is once again useful for recursive globbing with ``**`` in Python 192 | 3.5+. 193 | 194 | One very useful feature glob literals in xonsh is that they can be used 195 | to return pathlib.Path_ instances, which are a very pleasant way of 196 | dealing with paths if I do say so myself. This is done by prefixing 197 | either type of glob string with a ``p`` 198 | 199 | .. code:: bash 200 | 201 | >>> for p in p`/etc/.*`: 202 | ... if p.is_dir(): 203 | ... print(p) 204 | ... 205 | /etc/ImageMagick-6 206 | /etc/ImageMagick-7 207 | /etc/NetworkManager 208 | /etc/UPower 209 | /etc/X11 210 | /etc/asciidoc 211 | /etc/audisp 212 | /etc/audit 213 | [...] 214 | 215 | .. _regex globbing: 216 | https://xon.sh/tutorial.html#advanced-path-search-with-backticks 217 | .. _pathlib.Path: 218 | https://docs.python.org/3/library/pathlib.html#basic-use 219 | 220 | Environment Variables 221 | ~~~~~~~~~~~~~~~~~~~~~ 222 | In xonsh, "environment variables" are prefixed with a ``$``, as in Bash. 223 | xonsh's notion of environment variables includes things like ``$HOME`` 224 | and ``$PATH``, but also includes the assignment of arbitrary values to 225 | arbitrary names beginning with ``$``, which only exist for the lifetime 226 | of the current shell. These values are global, and they work in both 227 | subprocess mode and Python mode. In subprocess mode, this is how they 228 | are converted into arguments: 229 | 230 | - certain built-in environment variables have predefined conversion 231 | functions, which will create a sensible string representation. 232 | - if a variable doesn't have such a function registered (e.g. any 233 | variable you create yourself), it will call ``str()`` on the object. 234 | 235 | An example of the first kind of variable is ``$PATH`` which is a wrapper 236 | on a list internally, but will print as colon-separated values (as a 237 | ``$PATH`` would in Bash). 238 | 239 | Environment variables work like any other variable in Python mode. Like 240 | Bash, these variables can be interpolated freely into strings. Unlike 241 | Bash, they don't require quoting for safety. 242 | 243 | .. code:: bash 244 | 245 | >>> for $p in p`/etc/.*`: 246 | ... if $p.is_dir(): 247 | ... echo '$p is a directory' 248 | ... 249 | /etc/ImageMagick-6 is a directory 250 | /etc/ImageMagick-7 is a directory 251 | /etc/NetworkManager is a directory 252 | /etc/UPower is a directory 253 | [...] 254 | 255 | Substitutions 256 | ~~~~~~~~~~~~~ 257 | 258 | Python Substitution 259 | +++++++++++++++++++ 260 | One problem with user-created environment variables is that they just 261 | call ``str()`` when they are used in subprocess mode. That means: 262 | 263 | .. code:: sh 264 | 265 | $ $dirs = ['/usr', '/bin', '/etc'] 266 | $ ls -ld $dirs 267 | /usr/bin/ls: cannot access '['\''/usr'\'', '\''/bin'\'', '\''/etc'\'']': No such file or directory 268 | 269 | The way to get this to do the right thing is with Python substitution. 270 | Python substitution allows embedding the value of arbitrary Python 271 | expressions into commands. If the Python value is an iterable, it will 272 | be split into separate arguments. Python substitution is marked with 273 | ``@()``. 274 | 275 | .. code:: sh 276 | 277 | $ dirs = ['/usr', '/bin', '/etc'] 278 | $ ls -ld @(dirs) 279 | lrwxrwxrwx 1 root root 7 Aug 21 16:21 /bin -> usr/bin 280 | drwxr-xr-x 1 root root 3068 Sep 25 22:47 /etc 281 | drwxr-xr-x 1 root root 80 Sep 25 19:43 /usr 282 | $ echo hello-@('foo bar baz'.split()) 283 | hello-foo hello-bar hello-baz 284 | $ # Cartesian products can also be produced 285 | $ echo @(list('abc')):@(list('def')) 286 | a:d a:e a:f b:d b:e b:f c:d c:e c:f 287 | 288 | Python substitution only works in subprocess mode (because it is 289 | redundant in Python mode). 290 | 291 | Command Substitution(s) 292 | +++++++++++++++++++++++ 293 | xonsh has two forms of command substitution. The first is similar to 294 | that of Bash, using ``$()`` syntax. 295 | 296 | .. code:: shell 297 | 298 | $ ls -l $(which vi) 299 | lrwxrwxrwx 1 root root 4 Feb 27 2018 /usr/bin/vi -> nvim 300 | $ # why are permissions on this alias set to 777 instead of 755? 301 | $ # Oh well... 302 | 303 | If this form of substitution is used in Python mode, it returns a 304 | string. 305 | 306 | .. code:: sh 307 | 308 | $ print(repr($(which vi))) 309 | '/usr/bin/vi' 310 | 311 | The other form of command substitution only works in Python mode, where 312 | it returns a ``CommandPipeline`` object, which among other things, 313 | implements an iterator that lazily yields lines as they become available 314 | from the process. Trailing newlines are not stripped. 315 | 316 | .. code:: python 317 | 318 | >>> for line in !(ls): 319 | ... print(line.split()) 320 | ... 321 | ['total', '40'] 322 | ['-rw-r--r--', '1', 'ninjaaron', 'ninjaaron', '26872', 'Oct', '3', '23:01', 'out.html'] 323 | ['-rw-r--r--', '1', 'ninjaaron', 'ninjaaron', '10726', 'Oct', '3', '23:20', 'README.rst'] 324 | 325 | This object has other interesting properties as well, such as boolean 326 | coercion based on the exit code of the process. Look at the 327 | documentation_ for further details. This form of substitution is 328 | probably what you generally want in Python mode. 329 | 330 | You can also force subprocess mode without capturing output using 331 | ``$[]`` and ``![]``. ``![]`` returns a (weirdly unprintable) object with 332 | information about the process. ``$[]`` always returns ``None``, and I 333 | don't know why anyone would ever use it over ``![]``. 334 | 335 | .. _documentation: 336 | https://xon.sh/tutorial.html#captured-subprocess-with-and 337 | 338 | Basic Configuration, etc. 339 | ------------------------- 340 | Information on configuration is spread out all over the official docs, 341 | which was the most frustrating thing for me when I was trying it out the 342 | first time. I've tried to collect them here. 343 | 344 | - `Run Control File`_ tells about ~/.xonshrc and has some things you 345 | might want to stick in it. 346 | - `Customizing xonsh`_ also shows how to do some interesting things like 347 | set the color scheme, but also some bad things like how to set xonsh 348 | as your default shell. Believe it or not, some 3rd-party programs do 349 | rely on the default shell setting, and setting your shell with 350 | ``chsh`` to a non-POSIX shell can break such programs. My advice is to 351 | instead configure your terminal to launch with xonsh running, rather 352 | than to change your default shell. 353 | - `Customizing the Prompt`_ 354 | - `Environment Variables`_ is a complete list of environment variables, 355 | many of which can be used for settings. 356 | - Aliases_ work differently in xonsh than in other shells. 357 | 358 | Personally, because I use several shells (zsh, fish and xonsh), I try to 359 | avoid more complex functions in my shell configuration files. Instead, I 360 | keep simple aliases in `one place`_ and parse out the correct code for 361 | different shells from there. This is how I do it in xonsh. 362 | 363 | .. code:: python 364 | 365 | from hashlib import md5 366 | # import shell aliases 367 | al_cache = p'$HOME/.cache/ali_cache.xsh' # home of xonsh aliases. 368 | al_path = p'$HOME/.aliases' # home of POSIX aliases. 369 | al_hash = p'$HOME/.cache/ali_hash' # place where a hash of the 370 | # POSIX aliases live. 371 | 372 | # see if the file has changed. Probably should just do this with timestamps. 373 | with al_path.open('rb') as af, al_hash.open('rb') as ah: 374 | old_hash = ah.read() 375 | shell_aliases = af.read() 376 | new_hash = md5(shell_aliases).digest() 377 | 378 | if old_hash != new_hash: 379 | # find lines containing aliases and reformate them to xonsh aliases. 380 | ali = '\n'.join(i for i in shell_aliases.decode().splitlines() 381 | if i.startswith('alias ')) 382 | ali = re.sub(r'^alias ([\w-]*)=(.*?)$', r"aliases['\1'] = \2", 383 | ali, flags=re.M) 384 | with al_hash.open('wb') as ah, al_cache.open('w') as ac: 385 | ah.write(new_hash) 386 | ac.write(ali) 387 | 388 | exec(ali) 389 | 390 | else: 391 | source @(al_cache) 392 | 393 | 394 | For things that cannot be expressed as simple aliases, I try to just 395 | write scripts so I don't have to worry about portability between my 396 | various exotic shells. 397 | 398 | My whole xonshrc is here_. It contains some strange and possibly bad 399 | ideas. It comes with no warranty. 400 | 401 | .. _Run Control File: https://xon.sh/xonshrc.html 402 | .. _Customizing xonsh: https://xon.sh/customization.html 403 | .. _Customizing the Prompt: https://xon.sh/tutorial.html#customizing-the-prompt 404 | .. _Environment Variables: https://xon.sh/envvars.html 405 | .. _Aliases: https://xon.sh/tutorial.html#aliases 406 | .. _one place: https://github.com/ninjaaron/dot/blob/master/dotfiles/aliases 407 | .. _here: https://github.com/ninjaaron/dot/blob/master/dotfiles/xonshrc 408 | 409 | Working with Python Virtual Environments 410 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 411 | I don't know if this is exactly part of configuration, but, as you 412 | might expect, the standard virtualenv tools don't work with xonsh. This 413 | may not exactly be a configuration thing, but it's something Python 414 | developers need to know, and the info about it is here: 415 | https://xon.sh/python_virtual_environments.html 416 | 417 | Using xonsh over SSH 418 | ~~~~~~~~~~~~~~~~~~~~ 419 | If you enjoy using xonsh and wish to use it on remote systems without 420 | having to install it everywhere you go, you may want to check out xxh_, 421 | a program for executing xonsh interactively on remote systems. 422 | 423 | .. _xxh: https://github.com/xonssh/xxh 424 | --------------------------------------------------------------------------------