├── .gitattributes ├── LICENSE ├── Lesson01 ├── Activity 1.txt ├── Activity 2.txt ├── Activity 3.txt ├── Activity 4.txt ├── Activity 5.txt ├── Exercise 1.txt ├── Exercise 2.txt ├── Exercise 3.txt ├── Exercise 4.txt ├── Exercise 5.txt ├── Exercise 6.txt ├── Exercise 7.txt └── Exercise 8.txt ├── Lesson02 ├── Activity 6.txt ├── Activity 7.txt ├── Exercise 10.txt ├── Exercise 11.txt ├── Exercise 12.txt └── Exercise 9.txt ├── Lesson03 ├── Activity 8.txt ├── Activity 9.txt ├── Exercise 13.txt ├── Exercise 14.txt └── Exercise 15.txt ├── Lesson04 ├── Activity 10 │ ├── pgn_extract_act1.sh │ ├── pgn_extract_act2.sh │ ├── pgn_extract_act3.sh │ ├── pgn_extract_act4.sh │ └── pgn_extract_act5.sh ├── Activity 11 │ ├── Activity 11.txt │ ├── taxi_act1.sh │ ├── taxi_act2.sh │ ├── taxi_act3.sh │ └── taxi_act4.sh ├── Exercise 16 │ └── Exercise 16.txt ├── Exercise 17 │ └── Exercise 17.txt ├── Exercise 18 │ ├── Exercise 18.txt │ └── pgn_extract1.sh ├── Exercise 19 │ ├── Exercise 19.txt │ └── pgn_extract2.sh ├── Exercise 20 │ ├── Exercise 20.txt │ ├── pgn_extract3.sh │ └── pgn_extract4.sh ├── Exercise 21 │ ├── Exercise 21.txt │ ├── pgn_extract5.sh │ ├── pgn_extract6.sh │ ├── pgn_extract7.sh │ ├── pgn_extract8.sh │ └── pgn_extract_final.sh ├── Exercise 22 │ ├── Exercise 22.txt │ ├── taxi1.sh │ ├── taxi2.sh │ ├── taxi3.sh │ ├── taxi4.sh │ ├── taxi5.sh │ └── taxi6.sh ├── Exercise 23 │ ├── Exercise 23.txt │ └── taxi7.sh └── Exercise 24 │ ├── Exercise 24.txt │ ├── taxi8.sh │ └── taxi9.sh ├── Lesson1.zip ├── Lesson2.zip ├── Lesson3.zip ├── Lesson4.zip ├── Packt-CLI.ova └── README.md /.gitattributes: -------------------------------------------------------------------------------- 1 | *.ova filter=lfs diff=lfs merge=lfs -text 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Training By Packt 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Lesson01/Activity 1.txt: -------------------------------------------------------------------------------- 1 | # Assuming that the working directory is the home directory of the user 2 | 3 | # Solution 1 4 | cd Lesson1/data 5 | cd pinaceae/cedrus/deodara/ 6 | less data.txt 7 | 8 | 9 | # Solution 2 10 | cd ../../.. 11 | cd pinaceae/abies/pindrow/ 12 | less data.txt 13 | # Type /derives from 14 | # Read the answer on that line 15 | 16 | 17 | # Solution 3 18 | cd ../../.. 19 | cd taxaceae/taxus/ 20 | tree -d 21 | 22 | 23 | # Solution 4 24 | cd ../.. 25 | tree 26 | 27 | 28 | 29 | 30 | -------------------------------------------------------------------------------- /Lesson01/Activity 2.txt: -------------------------------------------------------------------------------- 1 | # Assuming we start from the the home directory in each case, and Lesson1 folder is in it 2 | 3 | 4 | # Solution 5 | cd ~ 6 | cd Lesson1/ 7 | mkdir activity2 8 | cd data/taxaceae/torreya/ 9 | ls 10 | 11 | cp californica/data.txt ../../../activity2/ 12 | mv ../../../activity2/data.txt ../../../activity2/californica.txt 13 | 14 | cp fargesii/data.txt ../../../activity2/ 15 | mv ../../../activity2/data.txt ../../../activity2/fargesii.txt 16 | 17 | cp grandis/data.txt ../../../activity2/ 18 | mv ../../../activity2/data.txt ../../../activity2/grandis.txt 19 | 20 | cp jackii/data.txt ../../../activity2/ 21 | mv ../../../activity2/data.txt ../../../activity2/jackii.txt 22 | 23 | cp nucifera/data.txt ../../../activity2/ 24 | mv ../../../activity2/data.txt ../../../activity2/nucifera.txt 25 | 26 | cp parvifolia/data.txt ../../../activity2/ 27 | mv ../../../activity2/data.txt ../../../activity2/parvifolia.txt 28 | 29 | cp taxifolia/data.txt ../../../activity2/ 30 | mv ../../../activity2/data.txt ../../../activity2/taxifolia.txt 31 | 32 | 33 | cd ~ 34 | cd Lesson1/activity2/ 35 | ls -l -------------------------------------------------------------------------------- /Lesson01/Activity 3.txt: -------------------------------------------------------------------------------- 1 | # In this activity, You have to enter the a list of keystrokes and text to accomplish the task 2 | # Text in capital letters indicates a keystroke 3 | # The spaces seen between each text string or keystrokes should not be typed 4 | # example "HOME echo" means press the HOME key and immediately type "echo" without a space 5 | # Actual space characters are indicated as "SPACE" in capitals 6 | 7 | data.txt 8 | CTRL+W 9 | 10 | Pinaceae SPACE Cedrus SPACE Deodara 11 | HOME echo SPACE ALT+L DEL / ALT+L DEL / ALT+L / CTRL+Y ENTER 12 | 13 | Cupressaceae SPACE Thuja SPACE Aphylla 14 | HOME echo SPACE ALT+L DEL / ALT+L DEL / ALT+L / CTRL+Y ENTER 15 | 16 | Taxaceae SPACE Taxus SPACE Baccata 17 | HOME echo SPACE ALT+L DEL / ALT+L DEL / ALT+L / CTRL+Y ENTER 18 | 19 | Podocarpaceae SPACE Podocarpus SPACE Alba 20 | HOME echo SPACE ALT+L DEL / ALT+L DEL / ALT+L / CTRL+Y ENTER 21 | -------------------------------------------------------------------------------- /Lesson01/Activity 4.txt: -------------------------------------------------------------------------------- 1 | # Activity 4 - group files by extension 2 | 3 | cd ~/Lesson1/data1 4 | mkdir images 5 | mkdir binaries 6 | mkdir misc 7 | 8 | 9 | mv *.jpg *.jpeg *.png *.gif images/ 10 | mv *.a *.so *.so.* binaries/ 11 | mv *.* misc/ 12 | 13 | 14 | # Read the counts from each tree command 15 | tree images 16 | tree binaries 17 | tree misc 18 | -------------------------------------------------------------------------------- /Lesson01/Activity 5.txt: -------------------------------------------------------------------------------- 1 | # Activity 5 solution - Directory wildcards 2 | # Navigate to the trees data folder in Lesson1/data 3 | cd ~/Lesson1/data 4 | 5 | # Read the file counts for each tree command 6 | 7 | tree p*/?a* 8 | 9 | tree p*/?i*/?u* 10 | 11 | tree t*/t* 12 | 13 | -------------------------------------------------------------------------------- /Lesson01/Exercise 1.txt: -------------------------------------------------------------------------------- 1 | # Step 1 has no commands - ensure that a Lesson1 folder exists (here assumed to be within the home dir) 2 | 3 | 4 | # Step 2 5 | cd Lesson1 6 | ls 7 | 8 | 9 | # Step 3 10 | cd data 11 | ls 12 | 13 | 14 | # Step 4 15 | ls taxaceae podocarpaceae 16 | 17 | 18 | # Step 5 19 | ls -l --color 20 | 21 | 22 | # Step 6 23 | cd taxaceae 24 | tree -d 25 | 26 | # Step 7 27 | cd taxus 28 | cd - 29 | 30 | # Step 8 31 | cd ../../.. 32 | cd - 33 | 34 | # Step 9 35 | cd 36 | cd - 37 | 38 | # Step 10 39 | pwd 40 | 41 | 42 | # Step 11 43 | pushd taxus/baccata/ 44 | pushd ../sumatrana/ 45 | pushd ../../../pinaceae/cedrus/deodara/ 46 | 47 | 48 | # Step 12 49 | popd 50 | popd 51 | popd 52 | popd 53 | 54 | 55 | # Step 13 56 | cd taxus/baccata 57 | cat data.txt 58 | 59 | 60 | # Step 14 61 | ls -l 62 | less data.txt 63 | 64 | 65 | -------------------------------------------------------------------------------- /Lesson01/Exercise 2.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | cd Lesson1 3 | 4 | 5 | # Step 2 6 | mkdir animals 7 | cd animals 8 | mkdir canis 9 | mkdir canis/familiaris 10 | mkdir canis/lupus 11 | mkdir canis/lupus/lupus 12 | mkdir leopardus/colocolo/pajeros 13 | 14 | 15 | # Step 3 16 | mkdir -p leopardus/colocolo/pajeros 17 | mkdir --parents panthera/tigris 18 | mkdir panthera/leo 19 | 20 | 21 | # Step 4 22 | tree 23 | 24 | 25 | # Step 5 26 | rmdir canis/familiaris/ 27 | rmdir canis 28 | rmdir canis/lupus 29 | 30 | 31 | # Step 6 32 | rmdir canis/lupus/lupus 33 | rmdir -p canis/lupus 34 | 35 | 36 | # Step 7 37 | tree 38 | 39 | 40 | # Step 8 41 | mkdir -p canis/lupus/lupus 42 | mkdir -p canis/lupus/familiaris 43 | ls 44 | 45 | 46 | # Step 9 47 | touch canis/lupus/familiaris/dog.txt 48 | touch panthera/leo/lion.txt 49 | touch canis/lupus/lupus/wolf.txt 50 | touch panthera/tigris/tiger.txt 51 | touch leopardus/colocolo/pajeros/colocolo.txt 52 | tree 53 | 54 | 55 | # Step 10 56 | mkdir dogs 57 | cp canis/lupus/familiaris/dog.txt dogs/ 58 | cp canis/lupus/lupus/wolf.txt dogs/ 59 | tree 60 | 61 | 62 | # Step 11 63 | mkdir cats 64 | cp -r panthera cats 65 | tree 66 | 67 | 68 | # Step 12 69 | mkdir bigcats 70 | cp -r --verbose leopardus/ panthera/ bigcats 71 | tree bigcats 72 | 73 | 74 | # Step 13 75 | cd .. 76 | mv animals beasts 77 | cd beasts 78 | ls 79 | 80 | 81 | # Step 14 82 | mv dogs/dog.txt fido.txt 83 | ls 84 | mv fido.txt dogs/ 85 | 86 | 87 | # Step 15 88 | mv canis dogs 89 | tree dogs 90 | 91 | 92 | # Step 16 93 | mkdir panthers 94 | mv --verbose panthera panthers 95 | tree panthers 96 | 97 | 98 | # Step 17 99 | tree dogs 100 | 101 | 102 | # Step 18 103 | rm dogs/fido.txt 104 | rm dogs/wolf.txt 105 | rm dogs/canis/lupus/familiaris/dog.txt 106 | rm dogs/canis/lupus/lupus/wolf.txt 107 | tree dogs 108 | 109 | 110 | # Step 19 111 | ls 112 | rm -r dogs 113 | ls 114 | 115 | 116 | # Step 20 117 | rm -r -i panthers 118 | ls 119 | 120 | rm -r -I bigcats 121 | ls 122 | 123 | 124 | # Step 21 125 | rm -r -v panthers/ 126 | 127 | 128 | # Step 22 129 | cd .. 130 | ls 131 | rm -r beasts 132 | ls 133 | 134 | 135 | -------------------------------------------------------------------------------- /Lesson01/Exercise 3.txt: -------------------------------------------------------------------------------- 1 | # "Press" means press those keystrokes. See lesson content for details. 2 | 3 | 4 | # Step 1 5 | mkdir data2 6 | cd data2 7 | 8 | 9 | # Step 2 10 | # Press: ^R 11 | animals 12 | 13 | 14 | # Step 3 15 | # Press: ^R^R ENTER 16 | # The following mkdir command appears and is executed from history 17 | # mkdir animals 18 | cd animals 19 | 20 | 21 | # Step 4 22 | # Press: ESC < ^Sfa^S^S Enter 23 | # The following mkdir command appears and is executed from history 24 | # mkdir -p canis/lupus/familiaris 25 | 26 | 27 | # Step 5 28 | # Press: up arrow 29 | # The following mkdir command appears on the command line 30 | # mkdir -p canis/lupus/familiaris 31 | 32 | # Edit the last word to make it as follows 33 | # mkdir -p canis/lupus/lupus 34 | # Press: Enter to execute it 35 | 36 | 37 | 38 | 39 | 40 | -------------------------------------------------------------------------------- /Lesson01/Exercise 4.txt: -------------------------------------------------------------------------------- 1 | # This exercise uses only "echo" so working directory is not relevant 2 | # "Press" means press those keystrokes, see lesson content for details 3 | # The commands that result from pressing the keys are shown as comments with a # at the start of the line 4 | # These commands are filled in by the shell history and editing and you are not supposed to type them 5 | # Text that appears on lines without # are meant to be typed 6 | 7 | 8 | # Step 1 9 | echo one two three four five/six/seven 10 | 11 | 12 | # Step 2 13 | # Press: up arrow Alt+B Alt+B Alt+B 14 | thousand 15 | # Press: SPACE Enter 16 | # The following command executes 17 | # echo one two three four thousand five/six/seven 18 | 19 | 20 | # Step 3 21 | # Press: up arrow Alt+Backspace Alt+B Alt+B ^Y 22 | / 23 | # Press: Enter 24 | # The following command executes 25 | # echo one two three four thousand seven/five/six/ 26 | 27 | 28 | # Step 4 29 | # Press: up arrow Alt+B Alt+B Alt+B Alt+B Alt+D End ^Y Enter 30 | # The following command executes 31 | # echo one two three four seven/five/six/thousand 32 | 33 | 34 | # Step 5 35 | # Press: up arrow Alt+B Alt+B Alt+B ^K Alt+B ^Y 36 | / 37 | # Press: Enter 38 | # The following command executes 39 | # echo one two three four five/six/thousand/seven/ 40 | 41 | 42 | # Step 6 43 | # Press: up arrow Alt+B Alt+B Alt+B ^U Alt+F ^Y Home 44 | echo 45 | # Press: SPACE Enter 46 | # The following command executes 47 | # echo sixecho one two three four five//thousand/seven/ 48 | -------------------------------------------------------------------------------- /Lesson01/Exercise 5.txt: -------------------------------------------------------------------------------- 1 | # Make sure you're in the parent of the Lesson1 folder 2 | # "Press" means press those keystrokes, see lesson content for details 3 | # None of the auto completed commands are actually to be executed, 4 | # Cancel each completed command with ^C or clear the command line before proceeding to the next "Press" step 5 | 6 | 7 | # Step 1 8 | cd Lesson1/data2/animals 9 | 10 | 11 | # Step 2 12 | cd canis 13 | Press: / Tab Tab Tab 14 | 15 | 16 | # Step 3 17 | f 18 | Press: Tab 19 | 20 | -------------------------------------------------------------------------------- /Lesson01/Exercise 6.txt: -------------------------------------------------------------------------------- 1 | # "Press" means press those keystrokes, see lesson content for details 2 | # None of the auto completed commands are actually to be executed, 3 | # Cancel each completed command with ^C or clear the command line before proceeding to the next "Press" step 4 | 5 | 6 | # Step 1 7 | les 8 | # Press: Tab 9 | # The following command gets completed 10 | # less 11 | 12 | # Step 2 13 | rmd 14 | # Press: Tab 15 | # The following command appears 16 | # rmdir 17 | 18 | # Step 3 19 | g 20 | # Press: Tab Tab 21 | # You are prompted if you wish to see all possibilities 22 | n 23 | 24 | 25 | 26 | -------------------------------------------------------------------------------- /Lesson01/Exercise 7.txt: -------------------------------------------------------------------------------- 1 | # "Press" means press those keystrokes, see lesson content for details 2 | # None of the auto completed commands are actually to be executed, 3 | # Cancel each completed command with ^C or clear the command line before proceeding to the next "Press" step 4 | 5 | 6 | # Step 1 7 | ls --col 8 | # Press: Tab 9 | # The following command gets completed 10 | # ls --color 11 | 12 | # Step 2 13 | ls --re 14 | # Press: Tab Tab 15 | # The following command options are shown 16 | # --recursive --reverse 17 | # The text "ls --re" is still on the command line - proceed to next step 18 | 19 | # Step 3 20 | c 21 | # Press: Tab 22 | # The option --recursive is completed and we get the following command 23 | # ls --recursive 24 | 25 | -------------------------------------------------------------------------------- /Lesson01/Exercise 8.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | cd Lesson1/data1 3 | 4 | 5 | # Step 2 6 | ls *.gif 7 | 8 | 9 | # Step 3 10 | mkdir gif 11 | mv *.gif gif 12 | 13 | 14 | # Step 4 15 | ls *.gif 16 | 17 | 18 | # Step 5 19 | ls gif/ 20 | 21 | 22 | # Step 6 23 | mkdir jpeg 24 | mv *.jpeg *.jpg jpeg 25 | 26 | 27 | # Step 7 28 | ls *.jpeg *.jpg 29 | 30 | 31 | # Step 8 32 | ls jpeg 33 | 34 | 35 | # Step 9 36 | ls *.so.? 37 | 38 | 39 | # Step 10 40 | ls google*.* 41 | 42 | 43 | # Step 11 44 | ls a?c*.* 45 | 46 | 47 | # Step 12 48 | ls !(*.jpg) 49 | 50 | 51 | # Step 13 52 | mv gif/* . 53 | mv jpeg/* . 54 | rm -r gif jpeg 55 | -------------------------------------------------------------------------------- /Lesson02/Activity 6.txt: -------------------------------------------------------------------------------- 1 | # Make sure you are in the ~/Lesson2 folder 2 | 3 | tail -n+2 land.csv | sort >sorted.txt 4 | 5 | cat -n sorted.txt >numbered.txt 6 | 7 | cat -T numbered.txt | head -n3 8 | 9 | tr ',' '\t' tabbed.txt 10 | 11 | cat -T tabbed.txt | head -n3 12 | 13 | join -o1.4,1.5,1.3 tabbed.txt tabbed.txt >342.txt 14 | 15 | join -o1.5,1.4,1.2 tabbed.txt tabbed.txt >431.txt 16 | 17 | head 342.txt 18 | 19 | head 431.txt 20 | 21 | 22 | -------------------------------------------------------------------------------- /Lesson02/Activity 7.txt: -------------------------------------------------------------------------------- 1 | # Make sure you are in the ~/Lesson2 folder 2 | 3 | grep 'Australia' land.csv >australia.txt 4 | 5 | sort -o australia.txt -k4 -t, -n australia.txt 6 | 7 | wc -l australia.txt 8 | 9 | bc <<< 55/2 10 | 11 | tail -n+28 australia.txt | head -n1 12 | 13 | grep -w '1998' population.csv >pop.txt 14 | 15 | sort -o pop.txt -k4 -t, -n pop.txt 16 | 17 | wc -l pop.txt 18 | 19 | bc <<< 249/2 20 | 21 | head -n125 pop.txt | tail -n1 22 | -------------------------------------------------------------------------------- /Lesson02/Exercise 10.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | cd Lesson1/data 3 | 4 | 5 | # Step 2 6 | tree | less 7 | 8 | 9 | # Step 3 10 | ls | tr '[:lower:]' '[:upper:]' 11 | 12 | 13 | # Step 4 14 | tree | tr '[:lower:]' '[:upper:]' | less 15 | 16 | 17 | # Step 5 18 | cd pinaceae/nothotsuga/longibracteata/ 19 | sort test.txt 24 | less test.txt 25 | rm test.txt 26 | 27 | 28 | # Step 7 29 | cd - 30 | ls -R . nosuchthing |& less 31 | 32 | -------------------------------------------------------------------------------- /Lesson02/Exercise 11.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | cd Lesson1/data/ 3 | 4 | 5 | # Step 2 6 | ls -l | cat -n 7 | 8 | 9 | # Step 3 10 | ls | tr 'a-z' 'A-Z' 11 | 12 | 13 | # Step 4 14 | ls | tr 'aeiou' 'AEIOU' 15 | 16 | 17 | # Step 5 18 | cd 19 | cd Lesson2 20 | 21 | 22 | # Step 6 23 | less land.csv 24 | 25 | 26 | # Step 7 27 | grep -w 'Austria' austria.txt 49 | cut -d, -f3 year.txt 50 | cut -d, -f4 percent.txt 51 | 52 | 53 | # Step 13 54 | paste -d' ' percent.txt year.txt 55 | 56 | 57 | # Step 14 58 | paste -d' ' percent.txt year.txt | sort -n 59 | 60 | 61 | # Step 15 62 | less payroll.tsv 63 | cut -f3,4 payroll.tsv | grep -v 'First' | sort >names.tsv 64 | 65 | 66 | # Step 16 67 | wc -l names.tsv 68 | 69 | 70 | # Step 17 71 | uniq names.tsv | wc -l 72 | 73 | 74 | # Step 18 75 | uniq -u names.tsv | wc -l 76 | 77 | -------------------------------------------------------------------------------- /Lesson02/Exercise 12.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | # Since we are continuing from the previous exercise, please make sure you are in the ~/Lesson2 directory 3 | # If necessary use the command "cd ~/Lesson2" before continuing 4 | head -n5 population.csv 5 | 6 | 7 | # Step 2 8 | cut -f1 -d, population.csv >p1.txt 9 | cut -f3 -d, population.csv >p3.txt 10 | cut -f4 -d, population.csv >p4.txt 11 | 12 | 13 | # Step 3 14 | paste -d$'/\t' p1.txt p3.txt p4.txt >p134.txt 15 | head -n5 p134.txt 16 | 17 | 18 | # Step 4 19 | cut -f1 -d, land.csv >l1.txt 20 | cut -f3 -d, land.csv >l3.txt 21 | cut -f4 -d, land.csv >l4.txt 22 | paste -d$'/\t' l1.txt l3.txt l4.txt >l134.txt 23 | head -n 5 l134.txt 24 | 25 | 26 | # Step 5 27 | tail -n+2 l134.txt | sort >land.txt 28 | tail -n+2 p134.txt | sort >pop.txt 29 | 30 | 31 | # Step 6 32 | join -t$'\t' --check-order -o auto -e 'UNKNOWN' -a1 -a2 land.txt pop.txt |less 33 | 34 | 35 | # Step 7 36 | namecounts.txt 41 | 42 | 43 | # Step 9 44 | grep -w 'SMITH' namecounts.txt | less 45 | 46 | 47 | # Step 10 48 | grep -w 'SMITH' namecounts.txt | tac | head -n5 49 | 50 | -------------------------------------------------------------------------------- /Lesson02/Exercise 9.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | cd Lesson1/data 3 | ls -l 4 | 5 | 6 | # Step 2 7 | ls -l >dir.txt 8 | cat dir.txt 9 | 10 | 11 | # Step 3 12 | ls -l nonexistent >dir.txt 13 | ls -l 14 | 15 | 16 | # Step 4 17 | ls -l pinaceae/ >dir.txt 18 | ls -l taxaceae/ >>dir.txt 19 | cat dir.txt 20 | 21 | 22 | # Step 5 23 | ls -l nonexistent taxaceae 2>dir.txt 24 | cat dir.txt 25 | 26 | 27 | # Step 6 28 | ls pinaceae nosuchthing >out.txt 2>err.txt 29 | cat out.txt 30 | cat err.txt 31 | 32 | 33 | # Step 7 34 | ls pinaceae nothing &>dir.txt 35 | cat dir.txt 36 | 37 | 38 | # Step 8 39 | cat -n numbered.txt 40 | less numbered.txt 41 | 42 | 43 | # Step 9 44 | cat 45 | Hello 46 | Bye 47 | Press: ^D 48 | 49 | 50 | # Step 10 51 | cat <sorted.txt 6 | 7 | # Step 3 8 | time (sort payroll.tsv >sorted.txt ; sort -k2 payroll.tsv >sorted2.txt) 9 | 10 | 11 | # Step 4 12 | time (sort payroll.tsv >sorted.txt & sort -k2 payroll.tsv >sorted2.txt) 13 | 14 | 15 | # Step 5 16 | time (sort payroll.tsv >sorted.txt ; sort -k2 payroll.tsv >sorted2.txt ; sort -k3 payroll.tsv >sorted3.txt) 17 | time (sort payroll.tsv >sorted.txt & sort -k2 payroll.tsv >sorted2.txt & sort -k3 payroll.tsv >sorted3.txt) 18 | 19 | # Step 6 20 | grep -q 'WALDO' payroll.tsv && echo 'Found' 21 | 22 | # Step 7 23 | grep -q 'PINOCCHIO' payroll.tsv || echo 'Not Found' 24 | -------------------------------------------------------------------------------- /Lesson03/Exercise 14.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | cd ~/Lesson1/data 3 | 4 | # Step 2 5 | grep -m1 -w -E -R 'India|China' | less 6 | 7 | # Step 3 8 | grep -R --color -E '[0-9]{2}-[0-9]{2} m ' 9 | 10 | 11 | # Step 4 12 | (grep -q -E '^[a-z][a-z0-9_]{,9}$' <<< 'hunter2' && echo 'Valid') || echo 'Invalid' 13 | (grep -q -E '^[a-z][a-z0-9_]{,9}$' <<< 'hunter222' && echo 'Valid') || echo 'Invalid' 14 | 15 | 16 | # Step 5 17 | (grep -q -E '^[a-z][a-z0-9_]{,9}$' <<< '2hunter' && echo 'Valid') || echo 'Invalid' 18 | (grep -q -E '^[a-z][a-z0-9_]{,9}$' <<< 'Hunter22' && echo 'Valid') || echo 'Invalid' 19 | 20 | 21 | # Step 6 22 | (grep -q -E '^[a-z][a-z0-9_]{,9}$' <<< 'hunter%2' && echo 'Valid') || echo 'Invalid' 23 | (grep -q -E '^[a-z][a-z0-9_]{,9}$' <<< 'hunter12345678' && echo 'Valid') || echo 'Invalid' 24 | 25 | 26 | # Step 7 27 | (grep -q -w -E '^\([[:digit:]]{3}\)$' <<< '(123)' && echo 'Valid') || echo 'Invalid' 28 | 29 | 30 | # Step 8 31 | (grep -q -w -E '^\([[:digit:]]{3}\)\ [[:digit:]]{3}$' <<< '(123) 456' && echo 'Valid') || echo 'Invalid' 32 | 33 | 34 | # Step 9 35 | (grep -q -w -E '^\([[:digit:]]{3}\)\ [[:digit:]]{3}-[[:digit:]]{4}$' <<< '(123) 456-7890' && echo 'Valid') || echo 'Invalid' 36 | 37 | 38 | # Step 10 39 | cd ~/Lesson3 40 | 41 | 42 | # Step 11 43 | cat markdown.txt 44 | 45 | 46 | # Step 12 47 | grep --color -E '\*\*([a-z]+)\*\*' \1#g' \1#g' -e 's#_([a-z]+)_#\1#g' log.txt 97 | 98 | 99 | # Step 21 100 | sort -m log.txt <(sort 1.txt) <(sort 2.txt) <(sort 3.txt) >sortmerge.txt 101 | 102 | 103 | # Step 22 104 | ls *.txt 105 | rm *.txt 106 | rm -r ../Lesson3/test 107 | -------------------------------------------------------------------------------- /Lesson04/Activity 10/pgn_extract_act1.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # A blank line is a line containing (only) 0 or more whitespaces 4 | regex_blank="^[[:space:]]*$" 5 | 6 | function read_chunk() 7 | { 8 | while read -r line 9 | do 10 | # If the first argument is 1, print this line 11 | [[ $1 -eq 1 ]] && echo "$line" 12 | 13 | # Quit if line is blank 14 | [[ $line =~ $regex_blank ]] && return 0 15 | done 16 | 17 | # Quit with failure return code here since read could not get another line 18 | return 1 19 | } 20 | 21 | 22 | function show_nth_game() 23 | { 24 | local count=1 25 | local should_print=0 26 | 27 | # As long as count is less than or equal to N 28 | while [[ $count -le $1 ]] 29 | do 30 | # Have we reached the Nth game, if so should_print is set 31 | [[ $count -eq $1 ]] && should_print=1 32 | 33 | # Process the first set of lines in a game, and printing only if should_print is 1 34 | # Exit if the read_chunk returned non-zero 35 | read_chunk $should_print || return 36 | 37 | # Repeat for the second chunk 38 | read_chunk $should_print 39 | 40 | # Increment the count and repeat the whole process 41 | count=$(( count + 1 )) 42 | 43 | # Quit after printing the game 44 | [[ $should_print -eq 1 ]] && break 45 | done 46 | } 47 | 48 | 49 | # reads through the initial game data of PGN data 50 | # and sets the variable found to 1 if the specified regex matched any line 51 | # Returns nonzero exit code if end of stream was encountered 52 | function filter_game() 53 | { 54 | # Initially assume not found 55 | found=0 56 | 57 | while read -r line 58 | do 59 | # Set found to 1 if the line matches the filter 60 | [[ $line =~ $1 ]] && found=1 61 | 62 | # If line is blank, we have reached end of game attributes, but not eos, so return 0 63 | [[ $line =~ $regex_blank ]] && return 0 64 | done 65 | 66 | # If it gets here there is no more data, so return nonzero 67 | return 1 68 | } 69 | 70 | 71 | # Read all the lines of the PGN move list and concatenate them into the variable moves 72 | function read_moves() 73 | { 74 | # Clear the moves variable 75 | moves='' 76 | 77 | while read -r line 78 | do 79 | # Quit if line is blank 80 | [[ $line =~ $regex_blank ]] && return 0 81 | 82 | # Append the line to moves with space in between 83 | moves="${moves} ${line}" 84 | 85 | done 86 | 87 | # Quit with failure return code here since read could not get another line 88 | return 1 89 | } 90 | 91 | 92 | # counts the number of moves in the moves list of a PGN format game 93 | # Assumes that "moves" is a string containing the complete moves list 94 | # result is stored in num_moves 95 | function count_moves() 96 | { 97 | # split moves into an array with the '.' as delimiter 98 | IFS='.' 99 | local moves_arr=( $moves ) 100 | 101 | # The second last entry contains the last moves index 102 | # Use the negative array index feature of bash to get it 103 | local second_last=${moves_arr[@]: -2 : 1} 104 | 105 | # Now Second last contains something like "Kg1 Qg3+ 31" and the last word is the count 106 | IFS=' ' 107 | local second_last_arr=( $second_last ) 108 | 109 | # Again use negative array index to get the number into moves_count 110 | num_moves=${second_last_arr[@]: -1 : 1} 111 | } 112 | 113 | 114 | # Display the indexes of all games that have a win in fewer than the specified moves 115 | function show_games_won_in() 116 | { 117 | local index=1 118 | 119 | # While we have not reached the end of stream 120 | while filter_game '\[Result "(1-0|0-1)"\]' 121 | do 122 | # Check if this game has a result 123 | if [[ $found -eq 1 ]] 124 | then 125 | 126 | # Read and count the moves 127 | read_moves 128 | count_moves 129 | 130 | # Display the game index and moves if it has fewer than $1 moves 131 | [[ $num_moves -le $1 ]] && echo "Game $index: $num_moves moves" && echo "$moves" && echo 132 | 133 | else # No result, skip over the moves list 134 | 135 | read_chunk 0 136 | fi 137 | 138 | # Increment the index 139 | index=$(( index + 1 )) 140 | done 141 | } 142 | 143 | 144 | getopts 'm:n:' opt 145 | 146 | case $opt in 147 | m) 148 | echo Displaying games won in "$OPTARG" moves or less 149 | echo 150 | show_games_won_in "$OPTARG" 151 | ;; 152 | 153 | n) 154 | echo Displaying Game "#$OPTARG" 155 | echo 156 | show_nth_game "$OPTARG" 157 | ;; 158 | esac 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | -------------------------------------------------------------------------------- /Lesson04/Activity 10/pgn_extract_act2.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | 4 | function read_chunk() 5 | { 6 | while read -r line 7 | do 8 | # If the first argument is 1, print this line 9 | [[ $1 -eq 1 ]] && echo "$line" 10 | 11 | # Quit if line is blank 12 | [[ -z $line ]] && return 0 13 | done 14 | 15 | # Quit with failure return code here since read could not get another line 16 | return 1 17 | } 18 | 19 | 20 | function show_nth_game() 21 | { 22 | local count=1 23 | local should_print=0 24 | 25 | # As long as count is less than or equal to N 26 | while [[ $count -le $1 ]] 27 | do 28 | # Have we reached the Nth game, if so should_print is set 29 | [[ $count -eq $1 ]] && should_print=1 30 | 31 | # Process the first set of lines in a game, and printing only if should_print is 1 32 | # Exit if the read_chunk returned non-zero 33 | read_chunk $should_print || return 34 | 35 | # Repeat for the second chunk 36 | read_chunk $should_print 37 | 38 | # Increment the count and repeat the whole process 39 | count=$(( count + 1 )) 40 | 41 | # Quit after printing the game 42 | [[ $should_print -eq 1 ]] && break 43 | done 44 | } 45 | 46 | 47 | # reads through the initial game data of PGN data 48 | # and sets the variable found to 1 if the specified regex matched any line 49 | # Returns nonzero exit code if end of stream was encountered 50 | function filter_game() 51 | { 52 | # Initially assume not found 53 | found=0 54 | 55 | while read -r line 56 | do 57 | # Set found to 1 if the line matches the filter 58 | [[ $line =~ $1 ]] && found=1 59 | 60 | # If line is blank, we have reached end of game attributes, but not eos, so return 0 61 | [[ -z $line ]] && return 0 62 | done 63 | 64 | # If it gets here there is no more data, so return nonzero 65 | return 1 66 | } 67 | 68 | 69 | # Read all the lines of the PGN move list and concatenate them into the variable moves 70 | function read_moves() 71 | { 72 | # Clear the moves variable 73 | moves='' 74 | 75 | while read -r line 76 | do 77 | # Quit if line is blank 78 | [[ -z $line ]] && return 0 79 | 80 | # Append the line to moves with space in between 81 | moves="${moves} ${line}" 82 | 83 | done 84 | 85 | # Quit with failure return code here since read could not get another line 86 | return 1 87 | } 88 | 89 | 90 | # counts the number of moves in the moves list of a PGN format game 91 | # Assumes that "moves" is a string containing the complete moves list 92 | function count_moves() 93 | { 94 | # split moves into an array with the '.' as delimiter 95 | IFS='.' 96 | local moves_arr=( $moves ) 97 | 98 | # The second last entry contains the last moves index 99 | # Use the negative array index feature of bash to get it 100 | local second_last=${moves_arr[@]: -2 : 1} 101 | 102 | # Now Second last contains something like "Kg1 Qg3+ 31" and the last word is the count 103 | IFS=' ' 104 | local second_last_arr=( $second_last ) 105 | 106 | # Again use negative array index to get the number into num_moves 107 | num_moves=${second_last_arr[@]: -1 : 1} 108 | } 109 | 110 | 111 | # Display the indexes of all games that have a win in fewer than the specified moves 112 | function show_games_won_in() 113 | { 114 | local index=1 115 | 116 | # While we have not reached the end of stream 117 | while filter_game '\[Result "(1-0|0-1)"\]' 118 | do 119 | # Check if this game has a result 120 | if [[ $found -eq 1 ]] 121 | then 122 | 123 | # Read and count the moves 124 | read_moves 125 | count_moves 126 | 127 | # Display the game index and moves if it has fewer than $1 moves 128 | [[ $num_moves -le $1 ]] && echo "Game $index: $num_moves moves" && echo "$moves" && echo 129 | 130 | else # No result, skip over the moves list 131 | 132 | read_chunk 0 133 | fi 134 | 135 | # Increment the index 136 | index=$(( index + 1 )) 137 | done 138 | } 139 | 140 | 141 | getopts 'm:n:' opt 142 | 143 | case $opt in 144 | m) 145 | echo Displaying games won in "$OPTARG" moves or less 146 | echo 147 | show_games_won_in "$OPTARG" 148 | ;; 149 | 150 | n) 151 | echo Displaying Game "#$OPTARG" 152 | echo 153 | show_nth_game "$OPTARG" 154 | ;; 155 | esac 156 | 157 | 158 | 159 | 160 | 161 | 162 | 163 | -------------------------------------------------------------------------------- /Lesson04/Activity 10/pgn_extract_act3.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Reads a set of lines and a blank line 4 | # prints them if first argument is 1 5 | function read_chunk() 6 | { 7 | while read -r line 8 | do 9 | # If the first argument is 1, print this line 10 | (( $1 == 1 )) && echo "$line" 11 | 12 | # Quit if line is blank 13 | [[ -z $line ]] && return 0 14 | done 15 | 16 | # Quit with failure return code here since read could not get another line 17 | return 1 18 | } 19 | 20 | 21 | function show_nth_game() 22 | { 23 | local count=1 24 | local should_print=0 25 | 26 | # As long as count is less than or equal to N 27 | while (( count <= $1 )) 28 | do 29 | # Have we reached the Nth game, if so should_print is set 30 | (( count == $1 )) && should_print=1 31 | 32 | # Process the first set of lines in a game, and printing only if should_print is 1 33 | # Exit if the read_chunk returned non-zero 34 | read_chunk $should_print || return 35 | 36 | # Repeat for the second chunk 37 | read_chunk $should_print 38 | 39 | # Increment the count and repeat the whole process 40 | (( count ++ )) 41 | 42 | # Quit after printing the game 43 | (( should_print == 1 )) && break 44 | done 45 | } 46 | 47 | 48 | # reads through the initial game data of PGN data 49 | # and sets the variable found to 1 if the specified regex matched any line 50 | # Returns nonzero exit code if end of stream was encountered 51 | function filter_game() 52 | { 53 | # Initially assume not found 54 | found=0 55 | 56 | while read -r line 57 | do 58 | # Set found to 1 if the line matches the filter 59 | [[ $line =~ $1 ]] && found=1 60 | 61 | # If line is blank, we have reached end of game attributes, but not eos, so return 0 62 | [[ -z $line ]] && return 0 63 | done 64 | 65 | # If it gets here there is no more data, so return nonzero 66 | return 1 67 | } 68 | 69 | 70 | # Read all the lines of the PGN move list and concatenate them into the variable moves 71 | function read_moves() 72 | { 73 | # Clear the moves variable 74 | moves='' 75 | 76 | while read -r line 77 | do 78 | # Quit if line is blank 79 | [[ -z $line ]] && return 0 80 | 81 | # Append the line to moves with space in between 82 | moves="${moves} ${line}" 83 | 84 | done 85 | 86 | # Quit with failure return code here since read could not get another line 87 | return 1 88 | } 89 | 90 | 91 | # counts the number of moves in the moves list of a PGN format game 92 | # Assumes that "moves" is a string containing the complete moves list 93 | function count_moves() 94 | { 95 | # split moves into an array with the '.' as delimiter 96 | IFS='.' 97 | local moves_arr=( $moves ) 98 | 99 | # The second last entry contains the last moves index 100 | # Use the negative array index feature of bash to get it 101 | local second_last=${moves_arr[@]: -2 : 1} 102 | 103 | # Now Second last contains something like "Kg1 Qg3+ 31" and the last word is the count 104 | IFS=' ' 105 | local second_last_arr=( $second_last ) 106 | 107 | # Again use negative array index to get the number into moves_count 108 | num_moves=${second_last_arr[@]: -1 : 1} 109 | } 110 | 111 | 112 | # Display the indexes of all games that have a win in fewer than the specified moves 113 | function show_games_won_in() 114 | { 115 | local index=1 116 | 117 | # While we have not reached the end of stream 118 | while filter_game '\[Result "(1-0|0-1)"\]' 119 | do 120 | # Check if this game has a result 121 | if (( found == 1 )) 122 | then 123 | 124 | # Read and count the moves 125 | read_moves 126 | count_moves 127 | 128 | # Display the game index and moves if it has fewer than $1 moves 129 | (( num_moves <= $1 )) && echo "Game $index: $num_moves moves" && echo "$moves" && echo 130 | 131 | else # No result, skip over the moves list 132 | 133 | read_chunk 0 134 | fi 135 | 136 | # Increment the index 137 | (( index++ )) 138 | done 139 | } 140 | 141 | 142 | getopts 'm:n:' opt 143 | 144 | case $opt in 145 | m) 146 | echo Displaying games won in "$OPTARG" moves or less 147 | echo 148 | show_games_won_in "$OPTARG" 149 | ;; 150 | 151 | n) 152 | echo Displaying Game "#$OPTARG" 153 | echo 154 | show_nth_game "$OPTARG" 155 | ;; 156 | esac 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | -------------------------------------------------------------------------------- /Lesson04/Activity 10/pgn_extract_act4.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Reads a set of lines and a blank line 4 | # prints them if first argument is 1 5 | function read_chunk() 6 | { 7 | while read -r line 8 | do 9 | # If the first argument is 1, print this line 10 | (( $1 == 1 )) && echo "$line" 11 | 12 | # Quit if line is blank 13 | [[ -z $line ]] && return 0 14 | done 15 | 16 | # Quit with failure return code here since read could not get another line 17 | return 1 18 | } 19 | 20 | 21 | function show_nth_game() 22 | { 23 | local count=1 24 | local should_print=0 25 | 26 | # As long as count is less than or equal to N 27 | while (( count <= $1 )) 28 | do 29 | # Have we reached the Nth game, if so should_print is set 30 | (( count == $1 )) && should_print=1 31 | 32 | # Process the first set of lines in a game, and printing only if should_print is 1 33 | # Exit if the read_chunk returned non-zero 34 | read_chunk $should_print || return 35 | 36 | # Repeat for the second chunk 37 | read_chunk $should_print 38 | 39 | # Increment the count and repeat the whole process 40 | (( count ++ )) 41 | 42 | # Quit after printing the game 43 | (( should_print == 1 )) && break 44 | done 45 | } 46 | 47 | 48 | # reads through the initial game data of PGN data 49 | # and sets the variable found to 1 if the specified regex matched any line 50 | # Returns nonzero exit code if end of stream was encountered 51 | function filter_game() 52 | { 53 | # Initially assume not found 54 | found=0 55 | 56 | while read -r line 57 | do 58 | # Set found to 1 if the line matches the filter 59 | [[ $line =~ $1 ]] && found=1 60 | 61 | # If line is blank, we have reached end of game attributes, but not eos, so return 0 62 | [[ -z $line ]] && return 0 63 | done 64 | 65 | # If it gets here there is no more data, so return nonzero 66 | return 1 67 | } 68 | 69 | 70 | # Read all the lines of the PGN move list and concatenate them into the variable moves 71 | function read_moves() 72 | { 73 | # Clear the moves variable 74 | moves='' 75 | 76 | while read -r line 77 | do 78 | # Quit if line is blank 79 | [[ -z $line ]] && return 0 80 | 81 | # Append the line to moves with space in between 82 | moves="${moves} ${line}" 83 | 84 | done 85 | 86 | # Quit with failure return code here since read could not get another line 87 | return 1 88 | } 89 | 90 | 91 | # counts the number of moves in the moves list of a PGN format game 92 | # Assumes that "moves" is a string containing the complete moves list 93 | function count_moves() 94 | { 95 | # split moves into an array with the '.' as delimiter 96 | IFS='.' 97 | local moves_arr=( $moves ) 98 | 99 | # The second last entry contains the last moves index 100 | # Use the negative array index feature of bash to get it 101 | local second_last=${moves_arr[@]: -2 : 1} 102 | 103 | # Now Second last contains something like "Kg1 Qg3+ 31" and the last word is the count 104 | IFS=' ' 105 | local second_last_arr=( $second_last ) 106 | 107 | # Again use negative array index to get the number into moves_count 108 | num_moves=${second_last_arr[@]: -1 : 1} 109 | } 110 | 111 | 112 | # Display the indexes of all games that have a win in fewer than the specified moves 113 | function show_games_won_in() 114 | { 115 | local index=1 116 | 117 | # While we have not reached the end of stream 118 | while filter_game '\[Result "(1-0|0-1)"\]' 119 | do 120 | # Check if this game has a result 121 | if (( found == 1 )) 122 | then 123 | 124 | # Read and count the moves 125 | read_moves 126 | count_moves 127 | 128 | # Display the game index and moves if it has fewer than $1 moves 129 | (( num_moves <= $1 )) && echo "Game $index: $num_moves moves" && echo "$moves" && echo 130 | 131 | else # No result, skip over the moves list 132 | 133 | read_chunk 0 134 | fi 135 | 136 | # Increment the index 137 | (( index++ )) 138 | done 139 | } 140 | 141 | 142 | function usage() 143 | { 144 | echo PGN Game extractor tool 145 | echo ----------------------- 146 | echo "Usage:" 147 | echo " $0 -n N" 148 | echo " Displays the Nth game in the file" 149 | echo 150 | echo " $0 -m M" 151 | echo " Displays the games that were won in fewer than M moves" 152 | echo 153 | } 154 | 155 | 156 | getopts 'm:n:' opt 157 | 158 | case $opt in 159 | m) 160 | echo Displaying games won in "$OPTARG" moves or less 161 | echo 162 | show_games_won_in "$OPTARG" 163 | ;; 164 | 165 | n) 166 | echo Displaying Game "#$OPTARG" 167 | echo 168 | show_nth_game "$OPTARG" 169 | ;; 170 | 171 | *) 172 | usage 173 | 174 | esac 175 | -------------------------------------------------------------------------------- /Lesson04/Activity 10/pgn_extract_act5.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Reads a set of lines and a blank line 4 | # prints them if first argument is 1 5 | function read_chunk() 6 | { 7 | while read -r line 8 | do 9 | # If the first argument is 1, print this line 10 | (( $1 == 1 )) && echo "$line" 11 | 12 | # Quit if line is blank 13 | [[ -z $line ]] && return 0 14 | done 15 | 16 | # Quit with failure return code here since read could not get another line 17 | return 1 18 | } 19 | 20 | 21 | function show_nth_game() 22 | { 23 | local count=1 24 | local should_print=0 25 | 26 | # As long as count is less than or equal to N 27 | while (( count <= $1 )) 28 | do 29 | # Have we reached the Nth game, if so should_print is set 30 | (( count == $1 )) && should_print=1 31 | 32 | # Process the first set of lines in a game, and printing only if should_print is 1 33 | # Exit if the read_chunk returned non-zero 34 | # Print an error message (remember count is 1 more than the last game) 35 | if ! read_chunk $should_print 36 | then 37 | echo -e "Invalid game index - maximum is $(( count - 1 )) \n" 38 | return 39 | fi 40 | 41 | # Repeat for the second chunk 42 | read_chunk $should_print 43 | 44 | # Increment the count and repeat the whole process 45 | (( count ++ )) 46 | 47 | # Quit after printing the game 48 | (( should_print == 1 )) && break 49 | done 50 | } 51 | 52 | 53 | # reads through the initial game data of PGN data 54 | # and sets the variable found to 1 if the specified regex matched any line 55 | # Returns nonzero exit code if end of stream was encountered 56 | function filter_game() 57 | { 58 | # Initially assume not found 59 | found=0 60 | 61 | while read -r line 62 | do 63 | # Set found to 1 if the line matches the filter 64 | [[ $line =~ $1 ]] && found=1 65 | 66 | # If line is blank, we have reached end of game attributes, but not eos, so return 0 67 | [[ -z $line ]] && return 0 68 | done 69 | 70 | # If it gets here there is no more data, so return nonzero 71 | return 1 72 | } 73 | 74 | 75 | # Read all the lines of the PGN move list and concatenate them into the variable moves 76 | function read_moves() 77 | { 78 | # Clear the moves variable 79 | moves='' 80 | 81 | while read -r line 82 | do 83 | # Quit if line is blank 84 | [[ -z $line ]] && return 0 85 | 86 | # Append the line to moves with space in between 87 | moves="${moves} ${line}" 88 | 89 | done 90 | 91 | # Quit with failure return code here since read could not get another line 92 | return 1 93 | } 94 | 95 | 96 | # counts the number of moves in the moves list of a PGN format game 97 | # Assumes that "moves" is a string containing the complete moves list 98 | function count_moves() 99 | { 100 | # split moves into an array with the '.' as delimiter 101 | IFS='.' 102 | local moves_arr=( $moves ) 103 | 104 | # The second last entry contains the last moves index 105 | # Use the negative array index feature of bash to get it 106 | local second_last=${moves_arr[@]: -2 : 1} 107 | 108 | # Now Second last contains something like "Kg1 Qg3+ 31" and the last word is the count 109 | IFS=' ' 110 | local second_last_arr=( $second_last ) 111 | 112 | # Again use negative array index to get the number into moves_count 113 | num_moves=${second_last_arr[@]: -1 : 1} 114 | } 115 | 116 | 117 | # Display the indexes of all games that have a win in fewer than the specified moves 118 | function show_games_won_in() 119 | { 120 | local index=1 121 | local game_count=0 122 | # While we have not reached the end of stream 123 | while filter_game '\[Result "(1-0|0-1)"\]' 124 | do 125 | # Check if this game has a result 126 | if (( found == 1 )) 127 | then 128 | 129 | # Read and count the moves 130 | read_moves 131 | count_moves 132 | 133 | # Display the game index and moves if it has fewer than $1 moves 134 | # Also increment game_count to keep track if there were any games at all 135 | if (( num_moves <= $1 )) 136 | then 137 | (( ++game_count )) 138 | echo "Game $index: $num_moves moves" 139 | echo "$moves" 140 | echo 141 | fi 142 | else # No result, skip over the moves list 143 | read_chunk 0 144 | fi 145 | 146 | # Increment the index 147 | (( index++ )) 148 | done 149 | 150 | # Print message if no games were found 151 | (( game_count == 0 )) && echo -e "No games\n" 152 | } 153 | 154 | 155 | function usage() 156 | { 157 | echo PGN Game extractor tool 158 | echo ----------------------- 159 | echo "Usage:" 160 | echo " $0 -n N" 161 | echo " Displays the Nth game in the file" 162 | echo 163 | echo " $0 -m M" 164 | echo " Displays the games that were won in fewer than M moves" 165 | echo 166 | } 167 | 168 | 169 | getopts 'm:n:' opt 170 | 171 | case $opt in 172 | m) 173 | echo Displaying games won in "$OPTARG" moves or less 174 | echo 175 | show_games_won_in "$OPTARG" 176 | ;; 177 | 178 | n) 179 | echo Displaying Game "#$OPTARG" 180 | echo 181 | show_nth_game "$OPTARG" 182 | ;; 183 | 184 | *) 185 | usage 186 | 187 | esac 188 | 189 | -------------------------------------------------------------------------------- /Lesson04/Activity 11/Activity 11.txt: -------------------------------------------------------------------------------- 1 | # Activity 2 - Shell Scripting - NYC Taxi Trip Analysis 2 | # Make sure you are in the ~/Lesson4 folder 3 | 4 | # Solution 1 5 | ./taxi_act1.sh nyc_taxi2.csv 6 | 7 | # Solution 2 - Note the use of input redirection rather than an argument 8 | ./taxi_act2.sh $temp_file_fares 10 | 11 | # Extract the 4th column which has the distance in miles as decimal format 12 | # Since the field has a two digits of precision, we can just remove the '.' to get the distance in units of 1/100 of a mile 13 | cut -d, -f4 < "$1" | tr -d '.' >$temp_file_dists 14 | 15 | # read each line of the fares file and total it, deal with leading 0s 16 | IFS=' ' 17 | total_fare=0 18 | while read fare 19 | do 20 | (( total_fare += 10#${fare} )) 21 | done <$temp_file_fares 22 | 23 | 24 | # read each line of the distances file and total it, deal with leading 0 25 | total_dist=0 26 | while read dist 27 | do 28 | (( total_dist += 10#${dist} )) 29 | done <$temp_file_dists 30 | 31 | echo -n "Average fare per mile is: " 32 | echo -n $(bc <<< "scale=2; $total_fare / $total_dist ") 33 | echo '$' 34 | 35 | rm "$temp_file_fares" 36 | rm "$temp_file_dists" 37 | 38 | -------------------------------------------------------------------------------- /Lesson04/Activity 11/taxi_act2.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # read each line of the CSV file 4 | # Line has data like: 5 | # 2017-01-09 11:13:28,2017-01-09 11:25:45,1,3.30,15.3,737 6 | # using space and comma as IFS will get us the start and stop times at column index 2 and 4 7 | 8 | IFS=', ' 9 | count=0 10 | 11 | # We dont need the 1st and 3rd column, but we need to specify a variable anyway, so we call it dummy1 and dummy2 12 | while read dummy1 start dummy2 stop 13 | do 14 | # Since the format is HH:MM:SS for both the time values we can use string compares 15 | # we will compare against 6AM and 10AM 16 | [[ $start > '06:00:00' && $stop < '10:00:00' ]] && (( count++ )) 17 | done 18 | 19 | echo "Number of trips that started after 6AM and ended before 10AM is $count" 20 | -------------------------------------------------------------------------------- /Lesson04/Activity 11/taxi_act3.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Calculates the higest taxi fare per minute 4 | 5 | 6 | # get a temp file name for the fares 7 | temp_file_fares=temp${RANDOM} 8 | 9 | # The 5th column of the file is the fare in decimal format with 2 digits of precision 10 | # Extract it and remove the '.' character, to get the fare in cents 11 | # Also include the 6th column that is the trip duration in seconds 12 | cut -d, -f5,6 "$1" | tr -d '.' >$temp_file_fares 13 | 14 | 15 | # read each line of the temp file 16 | # format has FF,ZZ 17 | # where FF is the price values in cents and ZZ is the duration 18 | IFS=',' 19 | 20 | # this will hold the known maximum fare per unit time 21 | max=0 22 | 23 | # this will hold the line number of that max data 24 | line_no=0 25 | 26 | # Current line 27 | count=1 28 | 29 | while read fare duration 30 | do 31 | # Since we use integer divide, for precision, we scale up the fare by 100 and scale it down at the end 32 | cost=$(( (100 * 10#${fare} ) / 10#${duration} )) 33 | (( cost > max )) && max=$cost line_no=$count 34 | (( ++count )) 35 | done <$temp_file_fares 36 | 37 | # Remember that max has units of 1/100 cents per second, so the conversion factor to $/minute is 60/10000 38 | echo -n "Highest fare per minute is: " 39 | echo -n $(bc <<< "scale=2; $max*60/10000") 40 | echo '$' 41 | echo "Line: " $line_no 42 | 43 | rm "$temp_file_fares" 44 | 45 | -------------------------------------------------------------------------------- /Lesson04/Activity 11/taxi_act4.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # get a temp file name for the distances 4 | temp_file_dists=temp${RANDOM} 5 | 6 | # Extract the 4th column which has the distance in miles as decimal format 7 | # Since the field has a two digits of precision, we can just remove the '.' to get the distance in units of 1/100 of a mile 8 | # also include the duration that is in the last column 9 | cut -d, -f4,6 "$1" | tr -d '.' >$temp_file_dists 10 | 11 | # Maximum speed, line that contained it, and current line 12 | max_speed=0 13 | max_line_no=0 14 | count=1 15 | 16 | # read each line of the distances file split on comma 17 | IFS=',' 18 | while read -a line 19 | do 20 | duration=${line[1]} 21 | dist=${line[0]} 22 | # Multiply dist by 100 for precision (units become 1/100th of a mile) 23 | speed=$(( (10#${dist} * 100) / duration )) 24 | 25 | if (( speed > max_speed )) 26 | then 27 | max_speed=$speed 28 | max_line_no=$count 29 | fi 30 | 31 | (( count++ )) 32 | 33 | done <$temp_file_dists 34 | 35 | # Remember that max_speed has units of "1/10000 miles" per second, so the conversion to m/h is 3600/10000 36 | echo -n "Highest speed of any trip: " 37 | echo -n $(bc <<< "scale=2; ${max_speed}*3600/10000") 38 | echo ' miles/hour' 39 | 40 | echo "Line: " $max_line_no 41 | 42 | rm "$temp_file_dists" 43 | 44 | -------------------------------------------------------------------------------- /Lesson04/Exercise 16/Exercise 16.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | function ucase() 3 | { 4 | tr "[a-z]" "[A-Z]" <<< "$*" 5 | } 6 | 7 | ucase hello world 8 | 9 | # Step 2 to 4 10 | function hf() 11 | { 12 | local a=0 13 | local b=1 14 | local n=$1 15 | local i=0 16 | echo 0 17 | echo 1 18 | 19 | while [[ $i -lt $n ]] 20 | do 21 | c=$(( a + b )) 22 | echo $c 23 | a=$b 24 | b=$c 25 | i=$(( i + 1 )) 26 | done 27 | } 28 | 29 | hf 8 30 | 31 | 32 | # Step 5 to 6 33 | function greet() 34 | { 35 | local timestring='night' 36 | [[ $2 -ge 5 && $2 -le 11 ]] && timestring='morning' 37 | [[ $2 -ge 12 && $2 -le 15 ]] && timestring='afternoon' 38 | [[ $2 -ge 16 && $2 -le 21 ]] && timestring='evening' 39 | 40 | echo "Good ${timestring}, $1!" 41 | } 42 | 43 | greet Jack 5 44 | greet Jill 12 45 | greet Tom 16 46 | greet Mary 22 47 | -------------------------------------------------------------------------------- /Lesson04/Exercise 17/Exercise 17.txt: -------------------------------------------------------------------------------- 1 | # Step 1 2 | for i in {1..5}; do echo "$i: $RANDOM"; done 3 | 4 | # Step 2 5 | function random_digit() { echo $(( $RANDOM % 10 )) ;} 6 | for i in {1..5}; do random_digit; done 7 | 8 | # Step 3 9 | function read_digit { guess=''; read -N1 -p "Enter your guess: " guess; echo; } 10 | read_digit 11 | 12 | # Step 4 13 | function guess_it() 14 | { 15 | echo "I have thought of a random number between 0 and 9 - try to guess it in 5 tries" 16 | 17 | local answer=$(random_digit) 18 | local count=1 19 | 20 | while [[ $count -le 5 ]] 21 | do 22 | 23 | read_digit 24 | if [[ $guess =~ [0-9] ]] 25 | then 26 | if [[ $guess -eq $answer ]] 27 | then 28 | echo Correct answer 29 | return 0 30 | fi 31 | 32 | [[ $guess -lt $answer ]] && echo "Your guess is lower than what I thought of, try again!" 33 | [[ $guess -gt $answer ]] && echo "Your guess is higher than what I thought of, try again!" 34 | 35 | count=$(( count + 1 )) 36 | 37 | else 38 | echo "Please enter a digit!" 39 | fi 40 | done 41 | 42 | echo "Sorry, you used up 5 guesses!" 43 | echo "The number I had thought of was $answer" 44 | } 45 | 46 | # Step 5 47 | guess_it 48 | 49 | -------------------------------------------------------------------------------- /Lesson04/Exercise 18/Exercise 18.txt: -------------------------------------------------------------------------------- 1 | # Exercise 18: Chess Game Extractor - Parsing A PGN File 2 | # Make sure you are in the ~/Lesson4 folder 3 | 4 | # Step 1 to 3 - write following script (all the lines upto but not including Step 4) and save to pgn_extract1.sh 5 | # Before moving to the next step, remember to run the command chmod u+x on it to make it executable 6 | 7 | 8 | #!/usr/bin/env bash 9 | regex_blank="^[[:space:]]*$" 10 | while read -r line 11 | do 12 | [[ $1 -eq 1 ]] && echo "$line" 13 | [[ "$line" =~ $regex_blank ]] && break 14 | done 15 | 16 | # Step 4 - Test with blank line 17 | ./pgn_extract1.sh 0 18 | ./pgn_extract1.sh 1 19 | 20 | 21 | # Step 5 - Test with 1 nonblank and 1 blank line 22 | ./pgn_extract1.sh 0 23 | ./pgn_extract1.sh 1 24 | 25 | 26 | # Step 6 - Test with many nonblank and 1 blank line 27 | ./pgn_extract1.sh 0 28 | ./pgn_extract1.sh 1 29 | 30 | 31 | -------------------------------------------------------------------------------- /Lesson04/Exercise 18/pgn_extract1.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # A blank line is a line containing (only) 0 or more whitespaces 4 | regex_blank="^[[:space:]]*$" 5 | 6 | while read -r line 7 | do 8 | # If the first argument is 1, print this line 9 | [[ $1 -eq 1 ]] && echo "$line" 10 | 11 | # Quit if line is blank 12 | [[ "$line" =~ $regex_blank ]] && break 13 | done 14 | 15 | 16 | -------------------------------------------------------------------------------- /Lesson04/Exercise 19/Exercise 19.txt: -------------------------------------------------------------------------------- 1 | # Exercise 19: Chess Game Extractor - Extracting a Desired Game 2 | # Make sure you are in the ~/Lesson4 folder 3 | 4 | # Step 1 5 | function read_chunk() 6 | { 7 | while read -r line 8 | do 9 | [[ $1 -eq 1 ]] && echo "$line" 10 | [[ $line =~ $regex_blank ]] && return 0 11 | done 12 | 13 | return 1 14 | } 15 | 16 | 17 | # Step 2 to 6 - Write script and save to pgn_extract2.sh 18 | # Before moving to the next step, remember to run the command chmod u+x on it to make it executable 19 | 20 | 21 | # Step 7 22 | 23 | ./pgn_extract2.sh 2 test.txt 67 | 68 | 69 | # Step 14 - Create the taxi5.sh script as per the lesson 70 | # Before moving to the next step, remember to run the command chmod u+x on it to make it executable 71 | 72 | 73 | # Step 15 74 | time head -n5000 nyc_taxi.csv | ./taxi5.sh >test.txt 75 | 76 | 77 | # Step 16, 17 - Create the taxi6.sh script as per the lesson 78 | # Before moving to the next step, remember to run the command chmod u+x on it to make it executable 79 | 80 | 81 | # Step 18 82 | time head -n5000 nyc_taxi.csv | ./taxi6.sh >test.txt 83 | 84 | 85 | # Step 19 86 | ./taxi6.sh nyc_taxi2.csv 87 | head -n5 nyc_taxi2.csv 88 | 89 | 90 | # Step 20 91 | wc -l nyc_taxi.csv 92 | wc -l nyc_taxi2.csv 93 | -------------------------------------------------------------------------------- /Lesson04/Exercise 22/taxi1.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # This function takes two timestamp arguments in 'YYYY-MM-DD HH:MM:SS' format 4 | # Prints the time difference between the two values in seconds 5 | function trip_duration() 6 | { 7 | # Split the dates and times into an array, using space as the IFS 8 | IFS=' ' 9 | local dt_start=( $1 ) 10 | local dt_stop=( $2 ) 11 | 12 | # dt_start[1] and dt_stop contain the time value as HH:MM:SS 13 | # Split the times into HH, MM and SS, using colon as the IFS 14 | IFS=':' 15 | local t_start=( ${dt_start[1]} ) 16 | local t_stop=( ${dt_stop[1]} ) 17 | 18 | # Convert the HH MM SS to an absolute number of seconds 19 | local n_start=$(( t_start[0] * 3600 + t_start[1] * 60 + t_start[2] )) 20 | local n_stop=$(( t_stop[0] * 3600 + t_stop[1] * 60 + t_stop[2] )) 21 | 22 | echo $(( ((n_stop - n_start) + 86400) % 86400 )) 23 | } 24 | 25 | 26 | trip_duration "$1" "$2" 27 | -------------------------------------------------------------------------------- /Lesson04/Exercise 22/taxi2.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # This function takes two timestamp arguments in 'YYYY-MM-DD HH:MM:SS' format 4 | # Prints the time difference between the two values in seconds 5 | function trip_duration() 6 | { 7 | # Split the dates and times into an array, using space as the IFS 8 | IFS=' ' 9 | local dt_start=( $1 ) 10 | local dt_stop=( $2 ) 11 | 12 | # dt_start[1] and dt_stop contain the time value as HH:MM:SS 13 | # Split the times into HH, MM and SS, using colon as the IFS 14 | IFS=':' 15 | local t_start=( ${dt_start[1]} ) 16 | local t_stop=( ${dt_stop[1]} ) 17 | 18 | # Convert the HH MM SS to an absolute number of seconds 19 | local n_start=$(( t_start[0] * 3600 + t_start[1] * 60 + t_start[2] )) 20 | local n_stop=$(( t_stop[0] * 3600 + t_stop[1] * 60 + t_stop[2] )) 21 | 22 | echo $(( ((n_stop - n_start) + 86400) % 86400 )) 23 | } 24 | 25 | 26 | # read each line of the CSV 27 | while read -r line 28 | do 29 | # Split each line with a comma 30 | IFS=',' 31 | fields=( $line ) 32 | 33 | # The first two fields are the two time stamps 34 | trip_duration "${fields[0]}" "${fields[1]}" 35 | done 36 | 37 | -------------------------------------------------------------------------------- /Lesson04/Exercise 22/taxi3.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # This function takes two timestamp arguments in 'YYYY-MM-DD HH:MM:SS' format 4 | # Prints the time difference between the two values in seconds 5 | function trip_duration() 6 | { 7 | # Split the dates and times into an array, using space as the IFS 8 | IFS=' ' 9 | local dt_start=( $1 ) 10 | local dt_stop=( $2 ) 11 | 12 | # dt_start[1] and dt_stop contain the time value as HH:MM:SS 13 | # Split the times into HH, MM and SS, using colon as the IFS 14 | IFS=':' 15 | local t_start=( ${dt_start[1]} ) 16 | local t_stop=( ${dt_stop[1]} ) 17 | 18 | # Convert the HH MM SS to an absolute number of seconds 19 | local n_start=$(( 10#${t_start[0]} * 3600 + 10#${t_start[1]} * 60 + 10#${t_start[2]} )) 20 | local n_stop=$(( 10#${t_stop[0]} * 3600 + 10#${t_stop[1]} * 60 + 10#${t_stop[2]} )) 21 | 22 | echo $(( ((n_stop - n_start) + 86400) % 86400 )) 23 | } 24 | 25 | 26 | # read each line of the CSV 27 | while read -r line 28 | do 29 | # Split each line with a comma 30 | IFS=',' 31 | fields=( $line ) 32 | 33 | # The first two fields are the two time stamps 34 | trip_duration "${fields[0]}" "${fields[1]}" 35 | done 36 | 37 | -------------------------------------------------------------------------------- /Lesson04/Exercise 22/taxi4.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # This function takes two timestamp arguments in 'YYYY-MM-DD HH:MM:SS' format 4 | # Prints the time difference between the two values in seconds 5 | function trip_duration() 6 | { 7 | # Split the dates and times into an array, using space as the IFS 8 | IFS=' ' 9 | local dt_start=( $1 ) 10 | local dt_stop=( $2 ) 11 | 12 | # dt_start[1] and dt_stop contain the time value as HH:MM:SS 13 | # Split the times into HH, MM and SS, using colon as the IFS 14 | IFS=':' 15 | local t_start=( ${dt_start[1]} ) 16 | local t_stop=( ${dt_stop[1]} ) 17 | 18 | # Convert the HH MM SS to an absolute number of seconds 19 | local n_start=$(( 10#${t_start[0]} * 3600 + 10#${t_start[1]} * 60 + 10#${t_start[2]} )) 20 | local n_stop=$(( 10#${t_stop[0]} * 3600 + 10#${t_stop[1]} * 60 + 10#${t_stop[2]} )) 21 | 22 | echo $(( ((n_stop - n_start) + 86400) % 86400 )) 23 | } 24 | 25 | 26 | # read each line of the CSV 27 | while read -r line 28 | do 29 | # Split each line with a comma 30 | IFS=',' 31 | fields=( $line ) 32 | 33 | # The first two fields are the two time stamps 34 | duration=$(trip_duration "${fields[0]}" "${fields[1]}") 35 | 36 | if (( (duration > 120) && (duration < 10800) )) 37 | then 38 | echo "${line},${duration}" 39 | fi 40 | 41 | done 42 | -------------------------------------------------------------------------------- /Lesson04/Exercise 22/taxi5.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # This function takes two timestamp arguments in 'YYYY-MM-DD HH:MM:SS' format 4 | # Prints the time difference between the two values in seconds 5 | function trip_duration() 6 | { 7 | # Split the dates and times into an array, using space as the IFS 8 | IFS=' ' 9 | local dt_start=( $1 ) 10 | local dt_stop=( $2 ) 11 | 12 | # dt_start[1] and dt_stop contain the time value as HH:MM:SS 13 | # Split the times into HH, MM and SS, using colon as the IFS 14 | IFS=':' 15 | local t_start=( ${dt_start[1]} ) 16 | local t_stop=( ${dt_stop[1]} ) 17 | 18 | # Convert the HH MM SS to an absolute number of seconds 19 | local n_start=$(( 10#${t_start[0]} * 3600 + 10#${t_start[1]} * 60 + 10#${t_start[2]} )) 20 | local n_stop=$(( 10#${t_stop[0]} * 3600 + 10#${t_stop[1]} * 60 + 10#${t_stop[2]} )) 21 | 22 | duration=$(( ((n_stop - n_start) + 86400) % 86400 )) 23 | } 24 | 25 | 26 | # read each line of the CSV 27 | while read -r line 28 | do 29 | # Split each line with a comma 30 | IFS=',' 31 | fields=( $line ) 32 | 33 | # The first two fields are the two time stamps 34 | trip_duration "${fields[0]}" "${fields[1]}" 35 | 36 | if (( (duration > 120) && (duration < 10800) )) 37 | then 38 | echo "${line},${duration}" 39 | fi 40 | 41 | done 42 | -------------------------------------------------------------------------------- /Lesson04/Exercise 22/taxi6.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # This function takes two timestamp arguments in 'YYYY-MM-DD HH:MM:SS' format 4 | # Prints the time difference between the two values in seconds 5 | function trip_duration() 6 | { 7 | # Get the timestamps from the line 8 | local line="$*" 9 | 10 | hh1=${line:11:2} 11 | mm1=${line:14:2} 12 | ss1=${line:17:2} 13 | 14 | hh2=${line:31:2} 15 | mm2=${line:34:2} 16 | ss2=${line:37:2} 17 | 18 | # Convert the HH MM SS to an absolute number of seconds 19 | n_start=$(( 10#${hh1} * 3600 + 10#${mm1} * 60 + 10#${ss1} )) 20 | n_stop=$(( 10#${hh2} * 3600 + 10#${mm2} * 60 + 10#${ss2} )) 21 | 22 | duration=$(( ((n_stop - n_start) + 86400) % 86400 )) 23 | } 24 | 25 | 26 | IFS=',' 27 | 28 | # read each line of the CSV 29 | while read -r line 30 | do 31 | # Split each line with a comma 32 | fields=( $line ) 33 | 34 | # The first two fields are the two time stamps 35 | trip_duration "${fields[0]}" "${fields[1]}" 36 | 37 | if (( (duration > 120) && (duration < 10800) )) 38 | then 39 | echo "${line},${duration}" 40 | fi 41 | 42 | done 43 | -------------------------------------------------------------------------------- /Lesson04/Exercise 23/Exercise 23.txt: -------------------------------------------------------------------------------- 1 | # Taxi Trip Analysis - Calculating Average Trip Speed 2 | # Make sure you are in the ~/Lesson4 folder 3 | 4 | # Step 1 5 | cut -d, -f4,6 < nyc_taxi2.csv | tr -d '.' >test.txt 6 | head -n5 test.txt 7 | 8 | 9 | #Step 2 to 5 - Create the taxi7.sh script as per the lesson 10 | # Before moving to the next step, remember to run the command chmod u+x on it to make it executable 11 | 12 | 13 | # Step 6 14 | ./taxi7.sh nyc_taxi2.csv 15 | 16 | -------------------------------------------------------------------------------- /Lesson04/Exercise 23/taxi7.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | 4 | # get a temp file name 5 | temp_file=temp${RANDOM} 6 | 7 | # cut the 4th and 6th column - distance and duration, get rid of the decimal point 8 | cut -d, -f4,6 "$1" | tr -d '.' >$temp_file 9 | 10 | total_duration=0 11 | total_distance=0 12 | IFS=',' 13 | 14 | while read distance duration 15 | do 16 | ((total_duration += 10#${duration})) 17 | ((total_distance += 10#${distance})) 18 | 19 | done <$temp_file 20 | 21 | miles=$(bc <<< "scale=2; $total_distance / 100") 22 | hours=$(bc <<< "scale=2; $total_duration / 3600") 23 | 24 | echo "$miles miles in $hours hours" 25 | echo "Average speed is:" $(bc <<< "scale=2; $miles / $hours") "miles/hr" 26 | 27 | # Get rid of the temp file 28 | rm "$temp_file" 29 | -------------------------------------------------------------------------------- /Lesson04/Exercise 24/Exercise 24.txt: -------------------------------------------------------------------------------- 1 | # Taxi Trip Analysis - Calculating Fare 2 | # Make sure you are in the ~/Lesson4 folder 3 | 4 | # Step 1,3 - Create the taxi8.sh script as per lesson - 5 | # Before moving to the next step, remember to run the command chmod u+x on it to make it executable 6 | 7 | 8 | # Step 4 9 | time head -n5000 nyc_taxi2.csv | ./taxi8.sh 10 | 11 | 12 | # Step 5 13 | cut -d, -f5 < nyc_taxi2.csv | tr -d '.' >nyc_taxi_fare.csv 14 | 15 | 16 | # Step 6-9 - Edit the script as per lesson and save as taxi9.sh 17 | 18 | 19 | # Step 10 20 | time ./taxi9.sh nyc_taxi2.csv 21 | -------------------------------------------------------------------------------- /Lesson04/Exercise 24/taxi8.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # read each line of the CSV 4 | total_fare=0 5 | count=0 6 | IFS=',' 7 | 8 | while read -a line 9 | do 10 | fare=${line[4]} 11 | total_fare=$(bc <<< "scale=2; $fare + $total_fare") 12 | (( count++ )) 13 | 14 | done 15 | 16 | echo -n "Average fare is: " 17 | echo -n $(bc <<< "scale=2; $total_fare / $count") 18 | echo '$' 19 | -------------------------------------------------------------------------------- /Lesson04/Exercise 24/taxi9.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | 4 | # get a temp file name 5 | temp_file=temp${RANDOM} 6 | 7 | cut -d, -f5 "$1" | tr -d '.' >$temp_file 8 | 9 | # read each line of the CSV 10 | total_fare=0 11 | count=0 12 | IFS=' ' 13 | 14 | while read fare 15 | do 16 | 17 | total_fare=$(( 10#${fare} + total_fare )) 18 | (( count++ )) 19 | 20 | done <$temp_file 21 | 22 | echo -n "Average fare is: " 23 | echo -n $(bc <<< "scale=2; ($total_fare / $count) / 100") 24 | echo '$' 25 | 26 | rm "$temp_file" 27 | -------------------------------------------------------------------------------- /Lesson1.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TrainingByPackt/Command-Line-Fundamentals/59801e4e4d6142a6c7e95755a265d16abdc20ad7/Lesson1.zip -------------------------------------------------------------------------------- /Lesson2.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TrainingByPackt/Command-Line-Fundamentals/59801e4e4d6142a6c7e95755a265d16abdc20ad7/Lesson2.zip -------------------------------------------------------------------------------- /Lesson3.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TrainingByPackt/Command-Line-Fundamentals/59801e4e4d6142a6c7e95755a265d16abdc20ad7/Lesson3.zip -------------------------------------------------------------------------------- /Lesson4.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TrainingByPackt/Command-Line-Fundamentals/59801e4e4d6142a6c7e95755a265d16abdc20ad7/Lesson4.zip -------------------------------------------------------------------------------- /Packt-CLI.ova: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:8024329b5ea1d5dcc1c531fb85b203794d85b4872b0136a0469c62174d59de88 3 | size 262288896 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![GitHub issues](https://img.shields.io/github/issues/TrainingByPackt/Command-Line-Fundamentals.svg)](https://github.com/TrainingByPackt/Command-Line-Fundamentals/issues) 2 | [![GitHub forks](https://img.shields.io/github/forks/TrainingByPackt/Command-Line-Fundamentals.svg)](https://github.com/TrainingByPackt/Command-Line-Fundamentals/network) 3 | [![GitHub stars](https://img.shields.io/github/stars/TrainingByPackt/Command-Line-Fundamentals.svg)](https://github.com/TrainingByPackt/Command-Line-Fundamentals/stargazers) 4 | [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/TrainingByPackt/Command-Line-Fundamentals/pulls) 5 | 6 | 7 | 8 | # Command Line Fundamentals 9 | From the Bash shell to traditional UNIX programs, and from redirection and pipes to automating tasks, Command Line Fundamentals teaches you all you need to know about how command lines work. The most basic interface to a computer, the command line, remains the most flexible and powerful way of processing data and performing and automating various day-to-day tasks. 10 | Command Line Fundamentals begins by exploring the basics and then focuses on the most common tool, the Bash shell (which is standard on all Linux and iOS systems). As you make your way through the course, you'll explore the traditional UNIX command-line programs implemented by the GNU project. You'll also learn how to use redirection and pipelines to assemble these programs to solve complex problems. Next, you'll learn how to use redirection and pipelines to assemble those programs to solve complex problems. 11 | By the end of this course, you'll have explored the basics of shell scripting, which will allow you to easily and quickly automate tasks. 12 | 13 | 14 | ## What you will learn 15 | * Use the Bash shell to run commands 16 | * Utilize basic Unix utilities such as cat, tr, sort, and uniq 17 | * Explore shell wildcards to manage groups of files 18 | * Apply useful keyboard shortcuts in shell 19 | * Employ redirection and pipes to process data 20 | * Write both basic and advanced shell scripts to automate tasks 21 | 22 | 23 | ### Hardware requirements 24 | For an optimal student experience, we recommend the following hardware configuration: 25 | * **Processor**: Any modern processor manufactured after 2010 26 | * **Memory**: 4 GB RAM 27 | * **Storage**: 4 GB available hard disk space 28 | 29 | 30 | ### Software requirements 31 | You’ll also need the following software installed in advance: 32 | 33 | * Operating System: Any desktop Linux version or iOS, or Windows XP or later versions 34 | 35 | 36 | ### Note: 37 | Since we are not dealing with source code, but interactive commands, each exercise has a text file with the commands for each step which can be directly pasted into a Bash shell to verify the results. 38 | 39 | * The exercises assume that the example user name is "robin", although this is not essential to their working. 40 | * Comments within the files are prefixed with " #", so that there are no issues if they get pasted into the terminal by mistake. 41 | * Please export HISTCONTROL=ignorespace so that the command history does not pick up any of the comments. 42 | * The command history needs to be maintained verbatim for some of the exercises - in other words some exercises depend on the commands that have been typed before in previous exercises 43 | * Each lesson has its own zip file of data to be extracted in the home directory 44 | * An Open Virtualization Appliance (built for use with Virtualbox) containing a Porteus Linux distro is provided in the repository 45 | * The VM image automatically downloads this repositories ZIP file and extracts it to the home directory of the guest user when it is first launched under the folders Snippets and Lesson1, Lesson2 etc. 46 | 47 | 48 | ### Important: 49 | 50 | The VM image is an OVA file of 250 MB stored via Git LFS - it is not included within the archive if you download the repository as a ZIP file from the github link. 51 | 52 | To get the file do either of the following: 53 | * Use `git clone git@github.com:TrainingByPackt/Command-Line-Fundamentals.git` to clone the repository, and git will download it along with the other files in the repository 54 | 55 | Since Packt Publishing has not been able to resolve the issue with github quota and they are not responsive to my communications - I have uploaded the OVA file here 56 | https://www.dropbox.com/s/ks9f7kcdsq7skhq/Packt-CLI.ova?dl=0 57 | 58 | 59 | 60 | 61 | --------------------------------------------------------------------------------