├── LICENSE.TXT ├── README.md ├── burp └── incexc │ ├── README.md │ ├── audio_compressed │ ├── generic_compressed │ ├── generic_excluded_extensions │ ├── generic_exclusions │ ├── image_compressed │ ├── linux_settings │ ├── macos_settings │ ├── std_settings │ ├── video_compressed │ ├── windows_program │ └── windows_settings ├── ddsplit.sh ├── emailCheck.sh ├── repairbadblocks.sh ├── ssh_jail.sh └── virsh ├── mk_vm.sh ├── vm_move.sh ├── vm_snapshots.sh └── vm_stats.sh /LICENSE.TXT: -------------------------------------------------------------------------------- 1 | Copyright (c) 2015-2019, Orsiris de Jong. ozy@netpower.fr 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | * Redistributions of source code must retain the above copyright 7 | notice, this list of conditions and the following disclaimer. 8 | * Redistributions in binary form must reproduce the above copyright 9 | notice, this list of conditions and the following disclaimer in the 10 | documentation and/or other materials provided with the distribution. 11 | * Neither the name of the author nor the 12 | names of its contributors may be used to endorse or promote products 13 | derived from this software without specific prior written permission. 14 | 15 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 16 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY 19 | DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 20 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 21 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 22 | ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 23 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 24 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 25 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # linuxscripts 2 | Collection of useful scripts / tools / files for Linux 3 | 4 | ## Burp incexc rules 5 | A set of regex rules and generic settings for Backup and Restore Program (BURP) from Graham Keeling (http://burp.grke.org) 6 | 7 | ## emailCheck.sh 8 | 9 | Low tech tool to cleanup mailing lists from unwanted emails before making a mass mailing. 10 | Performs various checks on a list of email adresses: 11 | 12 | 1. Converts all addresses to lowercase 13 | 2. Checks address' compliance against RC822 14 | 3. Checks address' domain for known typos and corrects them (please help improve that list) 15 | 4. Checks if email domain has MX records 16 | 5. Checks if email user or domain is test / example / spam, rendering them ambiguous 17 | 18 | Usage: 19 | ``` 20 | emailCheck.sh /path/to/email_list 21 | ``` 22 | 23 | Base script reads one email per line from input file. Script header contains instructions to read multicolumn CSV files. 24 | Warning: Using files comming from windows need prior conversion with dos2unix tool. 25 | 26 | ## ddsplit.sh 27 | 28 | Quick and dirty dd backup, useful to backup unknown file systems on fat32 / vfat devices. 29 | Performs disk backups via dd, compresses and splits into file chunks. 30 | Restores the splitted files to disk. 31 | Works especially well for unknown appliance filesystems (eg pfSense UFS which is not readable with standard Linux tools) 32 | 33 | Usage: 34 | ``` 35 | ddsplit.sh --backup /dev/sdX /mnt/myFile 1G 36 | ddsplit.sh --restore /mnt/ddsplit.1G.main.myFile.gz /dev/sdY 37 | ``` 38 | ## repairbadblocks.sh 39 | 40 | Highly experimental script to force SATA disk firmwares to reallocate a pending sector / bad sector using dd or hdparm. 41 | Use at your very own risk. Might even send your data to an unknown parallel universe where cows rule the world ! You have been warned. 42 | 43 | Usage: 44 | 45 | ``` 46 | repairbadblocks.sh /dev/sdX /tmp 47 | ``` 48 | 49 | ## ssh_jail.sh 50 | Creates a full ssh jail with basic commands like cp, mv, etc 51 | -------------------------------------------------------------------------------- /burp/incexc/README.md: -------------------------------------------------------------------------------- 1 | These are my personal burp incexc rules which: 2 | 3 | - Exclude already compressed files from burp zlib compression 4 | - Exclude a lot of temporary / lock / unuseful file extensions / Outlook offline cache from Exchange 5 | - Exclude some cloud Program cache like SkyDrive / DropBox and others 6 | - Exclude loads of Windows temp/cache/system files 7 | - Exclude browser caches 8 | - Exclude unnecessary Linux paths 9 | - Set standard settings (which you may have to modify to fit your needs) 10 | 11 | The regex are PCRE and validated by https://regexr.com 12 | 13 | In order to use it, simply write the following in the client config file server side: 14 | 15 | - For linux clients 16 | ``` 17 | . incexc/std_settings 18 | . incexc/linux_settings 19 | ``` 20 | 21 | - For windows clients 22 | ``` 23 | . incexc/std_settings 24 | . incexc/windows_settings 25 | ``` 26 | 27 | These rules are designed to exclude most unused system files from backup, but keep system programs and setting files (like Program Files & ProgramData or /sbin & /etc). 28 | It's better to backup too much than not enough :) 29 | 30 | All improvements are welcome. 31 | There may also exist multiple backup exclusion profiles like, whatever your needs are, eg: 32 | 33 | - windows_settings = Generic temp/cache/system path exclusions 34 | - windows_programs = Exclude most system paths (eg Windows / ProgramFiles / ProgramData) 35 | - linux_settings = Genreic path exclusions 36 | - linux_programs = Exclude /bin /sbin /usr/local/bin... 37 | 38 | Missing a MacOS guru that may write specific Mac settings file. 39 | -------------------------------------------------------------------------------- /burp/incexc/audio_compressed: -------------------------------------------------------------------------------- 1 | exclude_comp=aac 2 | exclude_comp=ac3 3 | exclude_comp=aif 4 | exclude_comp=aiff 5 | exclude_comp=asf 6 | exclude_comp=asx 7 | exclude_comp=cdda 8 | exclude_comp=cdr 9 | exclude_comp=flac 10 | exclude_comp=m4a 11 | exclude_comp=mkf 12 | exclude_comp=mp3 13 | exclude_comp=mpg3 14 | exclude_comp=mpga 15 | exclude_comp=oga 16 | exclude_comp=ogg 17 | exclude_comp=wma 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | -------------------------------------------------------------------------------- /burp/incexc/generic_compressed: -------------------------------------------------------------------------------- 1 | 2 | exclude_comp=7z 3 | exclude_comp=ace 4 | exclude_comp=apk 5 | exclude_comp=appx 6 | exclude_comp=appxbundle 7 | exclude_comp=arc 8 | exclude_comp=arj 9 | exclude_comp=b2 10 | exclude_comp=bhx 11 | exclude_comp=bz 12 | exclude_comp=bz2 13 | exclude_comp=cab 14 | exclude_comp=cpio 15 | exclude_comp=deb 16 | exclude_comp=dmg 17 | exclude_comp=jar 18 | exclude_comp=gpg 19 | exclude_comp=gz 20 | exclude_comp=gzip 21 | exclude_comp=hpk 22 | exclude_comp=hqx 23 | exclude_comp=jar 24 | exclude_comp=kgb 25 | exclude_comp=lzh 26 | exclude_comp=lha 27 | exclude_comp=lhz 28 | exclude_comp=lz 29 | exclude_comp=lzx 30 | exclude_comp=lzma 31 | exclude_comp=lzo 32 | exclude_comp=lzx 33 | exclude_comp=msi 34 | exclude_comp=pak 35 | exclude_comp=pit 36 | exclude_comp=rar 37 | exclude_comp=rpm 38 | exclude_comp=rz 39 | exclude_comp=rzip 40 | exclude_comp=s7z 41 | exclude_comp=sfark 42 | exclude_comp=sfx 43 | exclude_comp=sqz 44 | exclude_comp=tbz 45 | exclude_comp=tgz 46 | exclude_comp=txz 47 | exclude_comp=uu 48 | exclude_comp=uue 49 | exclude_comp=wdz 50 | exclude_comp=wim 51 | exclude_comp=xz 52 | exclude_comp=zip 53 | exclude_comp=zoo 54 | 55 | # Office documents are compressed too 56 | exclude_comp=accdb 57 | exclude_comp=accde 58 | exclude_comp=accdr 59 | exclude_comp=accdt 60 | exclude_comp=docm 61 | exclude_comp=docx 62 | exclude_comp=dotm 63 | exclude_comp=dotx 64 | exclude_comp=pptm 65 | exclude_comp=potm 66 | exclude_comp=potx 67 | exclude_comp=ppam 68 | exclude_comp=ppsx 69 | exclude_comp=pptx 70 | exclude_comp=sldx 71 | exclude_comp=sldm 72 | exclude_comp=thmx 73 | exclude_comp=xlsx 74 | exclude_comp=xlsm 75 | exclude_comp=xltx 76 | exclude_comp=xltm 77 | exclude_comp=xlsb 78 | exclude_comp=xlam 79 | -------------------------------------------------------------------------------- /burp/incexc/generic_excluded_extensions: -------------------------------------------------------------------------------- 1 | 2 | # Generic excluded extensions 3 | 4 | exclude_ext=back 5 | exclude_ext=bak 6 | exclude_ext=bkp 7 | exclude_ext=cache 8 | exclude_ext=chk 9 | exclude_ext=dmp 10 | exclude_ext=dump 11 | exclude_ext=err 12 | exclude_ext=lock 13 | exclude_ext=lockfile 14 | exclude_ext=log 15 | exclude_ext=log1 16 | exclude_ext=log2 17 | exclude_ext=old 18 | exclude_ext=tmp 19 | exclude_ext=temp 20 | 21 | # Browser not finished downloads 22 | exclude_ext=download 23 | exclude_ext=crdownload 24 | exclude_ext=part 25 | 26 | # Adobe lightroom preview files 27 | exclude_ext=lrprev 28 | 29 | # AutoCAD 30 | exclude_ext=dwl 31 | exclude_ext=dwl2 32 | exclude_ext=atmp 33 | 34 | # Microsoft Access lock file 35 | exclude_ext=laccdb 36 | exclude_ext=swp 37 | 38 | # Microsoft Outlook Exchange sync files 39 | #exclude_ext=ost 40 | 41 | # Microsoft Tracelog files 42 | exclude_ext=etl 43 | 44 | # Python compiled files 45 | exclude_ext=py[cod] 46 | 47 | -------------------------------------------------------------------------------- /burp/incexc/generic_exclusions: -------------------------------------------------------------------------------- 1 | # Microsoft Office lock files (eg */~$somefile.doc) 2 | exclude_regex = .*/~\$[^/]+\.[^/]+ 3 | 4 | # MacOS files 5 | exclude_regex = \.DS_Store 6 | 7 | # odrive sync dir 8 | exclude_regex = \.odrive 9 | 10 | # Generic directories / files 11 | exclude_regex = .*/[Cc]ache?[^/]+\.db 12 | exclude_regex = .*/TemporaryFiles 13 | 14 | # Generic Thumbs files 15 | exclude_regex = .*/([Tt]humb|[Ii]con)s?([Cc]ache)?.*\.db 16 | 17 | # Generic Thumbs directories 18 | exclude_regex = .*/\.?[Tt]humbnails 19 | 20 | # Generic tmp dir 21 | exclude_regex = .*/[Tt]e?mp$ 22 | exclude_regex = .*/[Tt]emporary$ 23 | 24 | # Generic cache dir 25 | exclude_regex = .*/([Ff]ont|[Ff]ile|[Dd]ist|[Nn]ative|[Pp]lay|[Pp]lugin|[Aa]sset|[Aa]ctivities|[Ss]cript|[Gg]pu|[Cc]ode|[Ll]ocal|[Ss]ession|[Ww]eb|JS|CRL)? ?[Cc]ache$ 26 | exclude_regex = .*/[Cc]ache[Ss]torage$ 27 | 28 | # Synology NAS working directory 29 | exclude_regex = .*/.SynologyWorkingDirectory 30 | -------------------------------------------------------------------------------- /burp/incexc/image_compressed: -------------------------------------------------------------------------------- 1 | exclude_comp=gif 2 | exclude_comp=jpg 3 | exclude_comp=jpeg 4 | exclude_comp=png 5 | exclude_comp=webp 6 | exclude_comp=tif 7 | exclude_comp=tiff 8 | -------------------------------------------------------------------------------- /burp/incexc/linux_settings: -------------------------------------------------------------------------------- 1 | # Standard linux system paths 2 | exclude_regex = ^/dev 3 | exclude_regex = .*/lost\+found 4 | exclude_regex = ^/media 5 | exclude_regex = ^/mnt 6 | exclude_regex = ^/proc 7 | exclude_regex = ^/run 8 | exclude_regex = ^/selinux 9 | exclude_regex = ^/sys 10 | exclude_regex = ^/tmp 11 | exclude_regex = ^/var/cache 12 | exclude_regex = ^/var/log 13 | exclude_regex = ^/var/run 14 | exclude_regex = ^/var/tmp 15 | 16 | exclude_fs=tmpfs 17 | 18 | # Linux cache files 19 | exclude_regex = \.cache 20 | exclude_regex = /\.cache/ 21 | 22 | # Generic home exclusions 23 | exclude_regex = ^/home/[^/]+/.debug 24 | exclude_regex = ^/home/[^/]+/.dbus 25 | exclude_regex = ^/home/[^/]+/.gvfs 26 | exclude_regex = ^/home/[^/]+/.local/share/gvfs-metadata 27 | exclude_regex = ^/home/[^/]+/.local/share/Trash 28 | exclude_regex = ^/home/[^/]+/.recently-used 29 | exclude_regex = ^/home/[^/]+/.thumbnails 30 | exclude_regex = ^/home/[^/]+/.xession-errors 31 | exclude_regex = ^/home/[^/]+/.Trash 32 | 33 | # Dropbox, OneDrive, SkyDrive data directories (not excluded by default because of cryptolockers attacks) 34 | #exclude_regex = ^/home/[^/]+/SkyDrive[^/]+ 35 | #exclude_regex = ^/home/[^/]+/Dropbox 36 | #exclude_regex = ^/home/[^/]+/OneDrive 37 | -------------------------------------------------------------------------------- /burp/incexc/macos_settings: -------------------------------------------------------------------------------- 1 | # Generic exclusions 2 | exclude_regex = /dev 3 | exclude_regex = /Network 4 | exclude_regex = /tmp 5 | exclude_regex = /cores 6 | exclude_regex = /afs 7 | exclude_regex = /automount 8 | exclude_regex = /private/Network 9 | exclude_regex = /private/tmp 10 | exclude_regex = /private/var/tmp 11 | exclude_regex = /private/var/folders 12 | exclude_regex = /private/var/run 13 | exclude_regex = /private/var/spool/postfix 14 | exclude_regex = /private/var/automount 15 | exclude_regex = /private/var/db/fseventsd 16 | exclude_regex = /Previous Systems 17 | exclude_regex = .*/lost+found 18 | 19 | exclude_regex = .*\.Trash.* 20 | exclude_regex = /\.vol/.* 21 | exclude_regex = .*/Network Trash Folder 22 | exclude_regex = .*\.fseventsd.* 23 | exclude_regex = *\.Spotlight-.* 24 | exclude_regex = .*Mobile.*Backups/.* 25 | 26 | -------------------------------------------------------------------------------- /burp/incexc/std_settings: -------------------------------------------------------------------------------- 1 | . audio_compressed 2 | . image_compressed 3 | . video_compressed 4 | . generic_compressed 5 | . generic_excluded_extensions 6 | . generic_exclusions 7 | 8 | nobackup=.nobackup 9 | atime=0 10 | scan_problem_raises_error=0 11 | -------------------------------------------------------------------------------- /burp/incexc/video_compressed: -------------------------------------------------------------------------------- 1 | exclude_comp=264 2 | exclude_comp=3fr 3 | exclude_comp=3g2 4 | exclude_comp=3gp 5 | exclude_comp=3gpp 6 | exclude_comp=asf 7 | exclude_comp=avi 8 | exclude_comp=avchd 9 | exclude_comp=divx 10 | exclude_comp=dv4 11 | exclude_comp=f4v 12 | exclude_comp=flv 13 | exclude_comp=mp4 14 | exclude_comp=mkv 15 | exclude_comp=h263 16 | exclude_comp=h264 17 | exclude_comp=h265 18 | exclude_comp=m4v 19 | exclude_comp=mjpeg 20 | exclude_comp=mkv 21 | exclude_comp=mov 22 | exclude_comp=movie 23 | exclude_comp=mpgv 24 | exclude_comp=mp4 25 | exclude_comp=mpeg 26 | exclude_comp=mpeg2 27 | exclude_comp=mpeg3 28 | exclude_comp=mpeg4 29 | exclude_comp=mpeg 30 | exclude_comp=mpg 31 | exclude_comp=mpv 32 | exclude_comp=ogv 33 | exclude_comp=qt 34 | exclude_comp=qtw 35 | exclude_comp=ram 36 | exclude_comp=rm 37 | exclude_comp=rmi 38 | exclude_comp=rmvb 39 | exclude_comp=swf 40 | exclude_comp=vob 41 | exclude_comp=webm 42 | exclude_comp=wmv 43 | exclude_comp=wmv3 44 | exclude_comp=video 45 | exclude_comp=xvid 46 | -------------------------------------------------------------------------------- /burp/incexc/windows_program: -------------------------------------------------------------------------------- 1 | # Exclusion list that removes most of Windows system and program files 2 | # This list is a complementary to windows_settings 3 | 4 | exclude_regex = ^[A-Z]:/Windows 5 | exclude_regex = ^[A-Z]:/Program Files 6 | exclude_regex = ^[A-Z]:/Program Files (x86) 7 | exclude_regex = ^[A-Z]:/ProgramData 8 | -------------------------------------------------------------------------------- /burp/incexc/windows_settings: -------------------------------------------------------------------------------- 1 | #### Generic Microsoft excludes 2 | 3 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Temp 4 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/LocalLow 5 | exclude_regex = ^[A-Z]:/Documents and Settings/[^/]+/Cookies 6 | exclude_regex = ^[A-Z]:/Documents and Settings/[^/]+/Recent 7 | exclude_regex = ^[A-Z]:/Documents and Settings/[^/]+/Local Settings/Temp 8 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Microsoft/Windows/Recent 9 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Temp 10 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/History 11 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Application Data 12 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Temporary Internet Files 13 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Internet Explorer 14 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/Temporary Internet Files 15 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/Caches 16 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/History 17 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Terminal Server Client/Cache 18 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/[^/]+/[Tt]e?mp 19 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Microsoft/Windows/Themes/CachedFiles 20 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/[^/]+/Internet Explorer 21 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/[^/]+/Microsoft/[^/]+/[^/]*[Cc]aches? 22 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/[^/]+/Microsoft/[^/]+/InetCookies 23 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/[^/]+/Microsoft Office/[^/]+OfficeFileCache 24 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/[^/]+/Microsoft/Office/Recent 25 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Outlook/RoamCache 26 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Office/[0-9\.]+/WebServiceCache 27 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Office/[0-9\.]+/Lync/Tracing 28 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/LocalLow/Microsoft/CryptnetUrlCache 29 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Downloaded Installations 30 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/GroupPolicy 31 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/AppV 32 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Messenger 33 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/OneNote 34 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Terminal Server Client 35 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/UEV 36 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows Live 37 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows Live Contacts 38 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Application Shortcuts 39 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Notifications 40 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/UsrClass\.dat\.LOG[0-9]+ 41 | exclude_regex = ^[A-Z]:/Users/[^/]+/ntuser\.dat\.LOG[0-9]+ 42 | 43 | # Temp folder for files that are ready to be burned on DVD 44 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/Burn 45 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/CD Burning 46 | 47 | # Generic application cache & temp folders (excludes all aaaCacheaaa or bbbTempbbb dirs) 48 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|LocalLow|Roaming)/[^/]+/[^/]*([Cc]ache|[Tt]emp)[^/]* 49 | 50 | # Various Win 10 caches 51 | exclude_regex = .*/(OfficeFile|SmartLookup|BackstageInAppNav|MruService)Cache 52 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/Caches 53 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/ActionCenterCache/ 54 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Feeds Cache 55 | 56 | # Error reports 57 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/WER 58 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/CrashDumps 59 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Diagnostics 60 | 61 | # Windows 10 Edge 62 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/[^/]*[Cc]ache 63 | exclude_regex = ^[A-Z]:/Users/[^/]+/MicrosoftEdgeBackups/backups 64 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/MicrosoftEdge/SharedCacheContainers 65 | 66 | # Windows 10 Store Application cache and state 67 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Packages/[^/]+/AC 68 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Packages/[^/]+/TempState 69 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Packages/[^/]+/LocalState 70 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Packages/[^/]+/LocalCache 71 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Packages/[^/]+/RoamingState 72 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Packages/[^/]+/AppData/User/Default/CacheStorage 73 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Packages/[^/]+/AppData/CacheStorage 74 | 75 | # Windows 10 various stuff 76 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/Notifications 77 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/Windows/Explorer 78 | 79 | # Windows downloads 80 | exclude_regex = ^[A-Z]:/Users/[^/]+/Downloads 81 | 82 | # Windows update cache 83 | exclude_regex = ^[A-Z]:/Windows/SoftwareDistribution/Download 84 | 85 | # Windows offline files 86 | exclude_regex = ^[A-Z]:/Windows/CSC 87 | 88 | exclude_regex = ^[A-Z]:/Windows/Temp 89 | exclude_regex = ^[A-Z]:/Windows/Downloaded Program Files 90 | exclude_regex = ^[A-Z]:/RECYCLER$ 91 | exclude_regex = ^[A-Z]:/\$recycle\.bin$ 92 | exclude_regex = ^[A-Z]:/System Volume Information$ 93 | # swap file (Windows XP, 7, 8) 94 | exclude_regex = ^[A-Z]:/pagefile\.sys$ 95 | # swap file?? (Windows 8) 96 | exclude_regex = ^[A-Z]:/swapfile\.sys$ 97 | exclude_regex = ^[A-Z]:/hiberfil\.sys$ 98 | # Windows temp installer 99 | exclude_regex = ^[A-Z]:/\$WINDOWS\.~BT$ 100 | # Windows 10 Upgrade previous install 101 | exclude_regex = ^[A-Z]:/Windows\.old$ 102 | exclude_regex = ^[A-Z]:/PerfLogs$ 103 | 104 | 105 | # Windows filesystem directories 106 | exclude_regex = ^[A-Z]:/\$mft$ 107 | exclude_regex = ^[A-Z]:/\$logfile$ 108 | exclude_regex = ^[A-Z]:/\$volume$ 109 | exclude_regex = ^[A-Z]:/\$bitmap$ 110 | exclude_regex = ^[A-Z]:/\$extend$ 111 | exclude_regex = ^[A-Z]:/\$reparse$ 112 | 113 | # Onedrive user executables 114 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Microsoft/OneDrive 115 | 116 | # Unnecessary folder exclusions 117 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Microsoft/Windows/Cookies 118 | exclude_regex = ^[A-Z]:/Users/[^/]+/NetHood 119 | exclude_regex = ^[A-Z]:/Users/[^/]+/PrintHood 120 | exclude_regex = ^[A-Z]:/Users/[^/]+/Cookies 121 | exclude_regex = ^[A-Z]:/Users/[^/]+/Recent 122 | exclude_regex = ^[A-Z]:/Users/[^/]+/SendTo 123 | exclude_regex = ^[A-Z]:/Users/[^/]+/LocalService 124 | exclude_regex = ^[A-Z]:/Users/[^/]+/NetworkService 125 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/LocalLow 126 | exclude_regex = ^[A-Z]:/Users/[^/]+/Tracing 127 | 128 | # Generic system file exlusions 129 | exclude_regex = ^[A-Z]:/*/MSOCache 130 | exclude_regex = ^[A-Z]:/*/Config\.Msi 131 | 132 | #### Applications 133 | 134 | # Office telemetry data 135 | exclude_regex = exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local//Microsoft/Office/OTeleData_.*.etl 136 | 137 | # Chrome 66+ 138 | # Chrome can also store in Roaming instead of Local 139 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/(Default|Profile [0-9]+)/[^/]*[Cc]ache 140 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/ShaderCache 141 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/(Default|Profile [0-9]+)/Session Storage 142 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/(Default|Profile [0-9]+)/Local Storage 143 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/(Default|Profile [0-9]+)/Extensions/Temp 144 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/(Default|Profile [0-9]+)/Service Worker/CacheStorage 145 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/(Default|Profile [0-9]+)/File System 146 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/(Default|Profile [0-9]+)/LOG 147 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/(Default|Profile [0-9]+)/.*/LOG 148 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/SwReporter 149 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Google/Chrome/User Data/PepperFlash 150 | 151 | # Opera 41+ 152 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Opera/Opera/profile/cache4 153 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Opera Software/Opera [^/]+/[^/]*[Cc]ache 154 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Opera Software/Opera [^/]+/Local Storage 155 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Opera Software/Opera [^/]+/Crash Reports 156 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Opera Software/Opera [^/]+/[^/]*[Cc]ache 157 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Opera Software/Opera [^/]+/Sessions 158 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Opera Software/Opera [^/]+/lockfile 159 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Opera Software/Opera [^/]+/LOG 160 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Opera Software/Opera [^/]+/.*/LOG 161 | 162 | # Vivaldi 1.x+ 163 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Vivaldi/Application 164 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Vivaldi/User Data/[^/]+/[^/]*[Cc]ache 165 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Vivaldi/User Data/[^/]+/Local Storage 166 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Vivaldi/User Data/[^/]+/LOG 167 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Vivaldi/User Data/[^/]+/.*/LOG 168 | 169 | # Firefox 44+ 170 | # There might exist a cache or a cache2 directory 171 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Mozilla/Firefox/Profiles/[^/]+/[^/]*[Cc]ache2? 172 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Mozilla/Firefox/Profiles/[^/]+/[Tt]humbnails 173 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Mozilla/Firefox/Crash Reports 174 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Mozilla/Firefox/Profiles/[^/]+/[^/]*[Cc]ache2 175 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Mozilla/Firefox/Profiles/[^/]+/sessionstore\.bak 176 | 177 | # Thunderbird 178 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Thunderbird/Mozilla Thunderbird/updates 179 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Thunderbird/Profiles/[^/]+/[Cc]ache[^/]* 180 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Thunderbird/Crash Reports 181 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Thunderbird/Profiles/[^/]+/crashes 182 | 183 | # Github Desktop 184 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/GitHubDesktop 185 | 186 | # Google Apps Sync 187 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Google/Google Apps Sync/Tracing 188 | 189 | # Adobe Acrobat DC 190 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Adobe/AcroCef/DC/Acrobat/Cache 191 | 192 | # Apple Logs 193 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Apple Computer/Logs 194 | 195 | # Apple iPhone backups :( 196 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Apple Computer/MobileSync/Backup 197 | # iTunes downloaded album artwork 198 | exclude_regex = ^[A-Z]:/Users/[^/]+/Music/iTunes/Album Artwork/Download 199 | 200 | # Java 201 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Sun 202 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(LocalLow|Roaming)/Sun/Java/Deployment/(cache|log|tmp) 203 | 204 | # Cisco Webex 205 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/WebEx/wbxcache 206 | 207 | # Ignite Realtime Spark client logs 208 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Roaming/Spark/logs 209 | 210 | # TeamViewer / SimpleHelp quick suppor 211 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/TeamViewer 212 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/JWrapper-Remote Support 213 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/JWrapper-SimpleHelp Technician 214 | 215 | # Zoom remote tool 216 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/(Local|Roaming)/Zoom 217 | 218 | # Dropbox, OneDrive, SkyDrive data directories (not excluded by default because of cryptolockers attacks) 219 | #exclude_regex = ^[A-Z]:/Users/[^/]+/SkyDrive[^/]+ 220 | #exclude_regex = ^[A-Z]:/Users/[^/]+/Dropbox 221 | #exclude_regex = ^[A-Z]:/Users/[^/]+/OneDrive 222 | 223 | # Dropbox config directory 224 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/Dropbox 225 | 226 | # Owncloud executables 227 | exclude_regex = ^[A-Z]:/Users/[^/]+/AppData/Local/ownCloud/.*\.exe 228 | 229 | #### Burp settings 230 | 231 | # Remove VSS info, also strips NTFS acls (protocol 1 only) 232 | strip_vss=1 233 | # Splitting VSS info would mean that for every file, 3 files exists (header, actual file, footer) 234 | split_vss=0 235 | -------------------------------------------------------------------------------- /ddsplit.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | PROGRAM="ddsplit.sh" # Quick and dirty command to backup / restore disk / data using dd and compress & split backup files 4 | AUTHOR="(L) 2016 by Orsiris de Jong" 5 | CONTACT="http://www.netpower.fr - ozy@netpower.fr" 6 | PROGRAM_VERSION=0.1-stable 7 | PROGRAM_BUILD=2016032302 8 | 9 | # Let dd error impact the whole pipe command 10 | set -o pipefail 11 | 12 | filesPrefix="ddsplit" 13 | filesPrefixLength=${#filesPrefix} 14 | 15 | function Backup { 16 | local dd_cmd_result=0 17 | local splitNumber=0 18 | local filenameSplit=main 19 | 20 | while [ "$dd_cmd_result" == 0 ] 21 | do 22 | cmd="dd if=\"$source\" bs=$splitSize count=1 skip=$splitNumber iflag=fullblock,direct | pigz --fast > \"$FILEPATH/$filesPrefix.$splitSize.$filenameSplit.$FILENAME.gz\"" 23 | echo "$cmd" 24 | eval "$cmd" 25 | dd_cmd_result=$? 26 | splitNumber=$((splitNumber + 1)) 27 | filenameSplit=$splitNumber 28 | done 29 | } 30 | 31 | function Restore { 32 | local splitSize 33 | local splitNumber 34 | local fileToRestore 35 | local filenameSuffix 36 | 37 | if [ "${FILENAME:0:$filesPrefixLength}" != "$filesPrefix" ]; then 38 | echo "Source file does not seem to be a $PROGRAM generated file." 39 | exit 1 40 | fi 41 | 42 | # Remove prefix 43 | filenameSuffix="${FILENAME#$filesPrefix.*}" 44 | # Get splitsize 45 | splitSize="${filenameSuffix%%.*}" 46 | # Remove split size 47 | filenameSuffix="${filenameSuffix#*.}" 48 | # Get split number 49 | splitNumber="${filenameSuffix%%.*}" 50 | # Remove split number 51 | filenameSuffix="${filenameSuffix#*.}" 52 | 53 | fileToRestore="$FILEPATH/$filesPrefix.$splitSize.$splitNumber.$filenameSuffix" 54 | while [ -f "$fileToRestore" ] 55 | do 56 | if [ "$splitNumber" == "main" ]; then 57 | splitNumber=0 58 | fi 59 | cmd="pigz -dc \"$fileToRestore\" | dd of=\"$destination\" bs=$splitSize seek=$splitNumber" 60 | echo "$cmd" 61 | eval "$cmd" 62 | splitNumber=$((splitNumber + 1)) 63 | filenameSplit=$splitNumber 64 | fileToRestore="$FILEPATH/$filesPrefix.$splitSize.$filenameSplit.$filenameSuffix" 65 | done 66 | } 67 | 68 | function CutFileNames { 69 | local filename="${1}" 70 | 71 | FILENAME="${filename##*/}" 72 | FILEPATH="${filename%/*}" 73 | if [ "$FILEPATH" == "" ] || [ "$FILEPATH" == "$FILENAME" ]; then 74 | FILEPATH="." 75 | fi 76 | } 77 | 78 | function Usage { 79 | echo "$PROGRAM - Low tech script to backup / restore with dd into compressed and splitted files" 80 | echo "$AUTHOR" 81 | echo "$CONTACT" 82 | echo "" 83 | echo "ATTENTION: This program may destroy all your data if used wrong. Use at your own risk !" 84 | echo "" 85 | echo "Usage:" 86 | echo "$PROGRAM --backup [source] [destination] [splitsize]" 87 | echo " Produces files called $filesPrefix.splitsize.splitnumber.destination.gz" 88 | echo " splitsize is optional and works just like dd does (eg 1K, 1M, 1G...). Maximum is 1G (default value is 1G)." 89 | echo "$PROGRAM --restore [source] [destination]" 90 | echo " Source needs to be the first split file called $filesPrefix.splitsize.master.somename.gz" 91 | exit 128 92 | } 93 | 94 | command="$1" 95 | source="$2" 96 | destination="$3" 97 | splitSize="${4:-1G}" 98 | 99 | if ([ "$source" == "" ] || [ "$destination" == "" ]); then 100 | Usage 101 | fi 102 | 103 | if [ "$command" == "--backup" ]; then 104 | CutFileNames "$destination" 105 | Backup 106 | fi 107 | 108 | if [ "$command" == "--restore" ]; then 109 | CutFileNames "$source" 110 | Restore 111 | fi 112 | 113 | if [ "$command" == "--version" ] ||[ "$command" == "-v" ]; then 114 | Usage 115 | fi 116 | -------------------------------------------------------------------------------- /emailCheck.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | PROGRAM="emailCheck.sh" 4 | AUTHOR="(L) 2014-2019 by Orsiris de Jong" 5 | CONTACT="http://www.netpower.fr/ - ozy@netpower.fr" 6 | PROGRAM_VERSION=0.6.4 7 | PROGRAM_BUILD=2019032801 8 | 9 | ## Email correction script 10 | ## Lowers all characters of email 11 | ## Checks if email format is valid againts RFC822 12 | ## Checks if there are any known domain typos 13 | ## Checks if email domain has valid MX records 14 | ## Checks if email address is ambiguous 15 | 16 | ## Warning: when processing files command windows, use dos2unix first to convert carriage return chars 17 | 18 | 19 | ###################################################################################################### Input file format options 20 | 21 | ## Example: File with only one mail address per line 22 | #CSV_EMAIL_IS_FIRST_COLUMN=true 23 | #CSV_INPUT_DELIMITER=$IFS 24 | #CSV_READ='email' 25 | #CSV_WRITE='email' 26 | 27 | ## Example: CSV file with three columns where email is in second column, where output CSV is comma instead of semicolon 28 | #CSV_EMAIL_IS_FIRST_COLUMN=false 29 | #CSV_INPUT_DELIMITER=';' 30 | #CSV_READ='col1 email col3' 31 | #CSV_WRITE='$col1,$email,$col3' 32 | 33 | CSV_EMAIL_IS_FIRST_COLUMN=true 34 | CSV_INPUT_DELIMITER=$IFS 35 | CSV_READ='email' 36 | CSV_WRITE='$email' 37 | 38 | ###################################################################################################### Input file format options 39 | 40 | # Check if internet is working by sending a ping to the following address 41 | inet_addr_to_test=www.google.com 42 | 43 | # Filename prefixes 44 | TMP_PREFIX="tmp" 45 | VALID_PREFIX="valid" 46 | NON_RFC_COMPLIANT_PREFIX="rfc_non_compliant" 47 | MISSING_MX_PREFIX="missing_mx" 48 | AMBIGUOUS_PREFIX="ambiguous" 49 | 50 | 51 | 52 | 53 | 54 | ## NO NEED TO EDIT UNDER THIS LINE 55 | 56 | # Initial counter values 57 | INCORRECT_MAILS=0 58 | INCORRECT_DOMAINS=0 59 | INCORRECT_MX=0 60 | AMBIGUOUS_MAILS=0 61 | VALID_MAILS=0 62 | 63 | # Lowers all characters 64 | function lowercase { 65 | local string="${1}" 66 | 67 | echo "$(echo $string | tr '[:upper:]' '[:lower:]')" 68 | } 69 | 70 | # Function checks if argument is valid against RFC822 71 | function checkRFC822 { 72 | local mail="${1}" 73 | local rfc822="^[a-z0-9!#\$%&'*+/=?^_\`{|}~-]+(\.[a-z0-9!#$%&'*+/=?^_\`{|}~-]+)*@([a-z0-9]([a-z0-9-]*[a-z0-9])?\.)+[a-z0-9]([a-z0-9-]*[a-z0-9])?\$" 74 | 75 | if [[ $mail =~ $rfc822 ]]; then 76 | return 0 77 | else 78 | return 1 79 | fi 80 | } 81 | 82 | # Function fixes some known typos in email domain parts 83 | function checkDomains { 84 | local mail="${1}" 85 | 86 | declare -a invalid_domains=( .neuf.fr neuf.com neuf.fre neuf.frr neuf neuf.fe neuf.ff 87 | .wanadoo.fr wanado.fr wanado.com wanadoo.fre anadoo.fr wanaoo.fr wanadoo wanadoo.com wanadoo.frr wanadoo wanadoo.fe wanadoo.ff 88 | .orange.fr ornge.fr orage.fr orage.com orange.com orange.frr orange orange.fe orange.ff 89 | .free.fr fre.fr fre.com free.com free.frr free.fe free.ff 90 | .club-internet.fr clubinternet.fr clubinternet.com club-internet.com club-internet clubinternet club_internet.fr club-internet.fe club-internet.ff 91 | .laposte.net laposte.com laposte 92 | .yahoo.fr yaho.fr yaho.com yaho.co.uk yahoo.frr yahoo.ciom yahoo.vom yahoo.c yahoo.cim yahoo.co yaoo.col yahoo.colm yahoo.con 93 | .sfr.fr sfr.fre sfr.frr sfr sfr.fe sfr.ff 94 | .hotmail.fr homail.fr hotail.fr hotmail.fe hotmail.ff homail.com hotail.com hotmail.ciom hotmail.vom hotmail.c hotmail.cim hotmail.co hotmail.col hotmail.colm hotmail.con 95 | .live.com life.fr live.frr live.fe live.ff life.com live.ciom live.vom live.c live.cim live.co live.col live.colm live.con 96 | .outlook.fr outlok.fr outlok.com outlook.ciom outlook.vom outlook.c outlook.cim outlook.co outlook.col outlook.colm outlook.con outhlook.fr outhlook.com 97 | .gmail.com gmail.fr gmal.fr gmail.frr gmail gmail.ciom g.mail gemail.com galml.com gmail.c gmail. gmail.cim gmail.clm gmail.co gmail.col gmail.comp gmail.con gmail.cpm gmail.de gmail.dk gmail.es gmail.org gmail.vom gmaill.com gmal.com gmeil.com gmail.vom gmail.colm 98 | .googlemail.com googlemail.con googlemail.cpm googlemail.de googlemail.fr googlemail.co.uk googlemail.es googlemail.dk googlemail.vom googlemail.c googlemail.cim googlemail.co googlemail.col googlemail.colm googlemail.con 99 | .aliceadsl.fr alice.fr aliseadsl.fr aliceadsl.com aliceadsl.frr aliceadsl aliceadsl.fe aliceadsl.ff 100 | .voila.fr voila.com voila.frr voila voila.fe voila.ff 101 | .skynet.be skynet.bee skynet 102 | .aol.fr aol.ciom aol.vom aol.c aol.cim aol.co aol.col aol.colm aol.con 103 | ) 104 | declare -a valid_domains=( neuf.fr neuf.fr neuf.fr neuf.fr neuf.fr neuf.fr neuf.fr 105 | wanadoo.fr wanadoo.fr wanadoo.fr wanadoo.fr wanadoo.fr wanaoo.fr wanadoo.fr wanadoo.fr wanadoo.fr wanadoo.fr wanadoo.fr wanadoo.fr 106 | orange.fr orange.fr orange.fr orange.fr orange.fr orange.fr orange.fr orange.fr orange.fr 107 | free.fr free.fr free.fr free.fr free.fr free.fr free.fr 108 | club-internet.fr club-internet.fr club-internet.fr club-internet.fr club-internet.fr club-internet.fr club-internet.fr club-internet.fr club-internet.fr 109 | laposte.net laposte.net laposte.net 110 | yahoo.fr yahoo.fr yahoo.com yahoo.co.uk yahoo.fr yahoo.com yahoo.com yahoo.com yahoo.com yahoo.com yahoo.com yahoo.com yahoo.com 111 | sfr.fr sfr.fr sfr.fr sfr.fr sfr.fr sfr.fr 112 | hotmail.fr hotmail.fr hotmail.fr hotmail.fr hotmail.fr hotmail.com hotmail.com hotmail.com hotmail.com hotmail.com hotmail.com hotmail.com hotmail.com hotmail.com hotmail.com 113 | live.com live.fr live.fr live.fr live.fr live.com live.com live.com live.com live.com live.com live.com live.com live.com 114 | outlook.fr outlook.fr outlook.com outlook.com outlook.com outlook.com outlook.com outlook.com outlook.com outlook.com outlook.com outlook.fr outlook.com 115 | gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com gmail.com 116 | googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com googlemail.com 117 | aliceadsl.fr aliceadsl.fr aliceadsl.fr aliceadsl.fr aliceadsl.frr aliceadsl aliceadsl.fr aliceadsl.fr 118 | voila.fr voila.fr voila.fr voila.fr voila.fr voila.fr 119 | skynet.be skynet.be skynet.be 120 | aol.fr aol.com aol.com aol.com aol.com aol.com aol.com aol.com aol.com 121 | ) 122 | 123 | 124 | local count=0 125 | 126 | # Dumb check of the number of elements per table that should match 127 | if [ ${#invalid_domains[@]} -ne ${#valid_domains[@]} ]; then 128 | echo "Bogus domain tables. Cannot continue." 129 | exit 1 130 | fi 131 | 132 | for i in "${invalid_domains[@]}"; do 133 | if [ "$i" == "${mail#*@}" ]; then 134 | mail="${mail%@*}@${valid_domains[$count]}" 135 | fi 136 | count=$((count + 1)) 137 | done 138 | 139 | # Function return 140 | echo "$mail" 141 | } 142 | 143 | # Function checks if MX records exist for the domain of an email address 144 | function checkMXDomains { 145 | local mail="${1}" 146 | 147 | if [ "$(dig "${mail#*@}" mx +short | wc -l)" -ne 0 ]; then 148 | return 0 149 | else 150 | return 1 151 | fi 152 | } 153 | 154 | function checkEnvironment { 155 | 156 | if ! type dig > /dev/null; then 157 | echo "This script needs dig to resolve MX records." 158 | exit 1 159 | fi 160 | 161 | if ! type tr > /dev/null; then 162 | echo "This script needs tr to transorm addresses to lowercase." 163 | exit 1 164 | fi 165 | 166 | echo "Checking for internet access." 167 | if [[ $(uname) == *"CYGWIN"* ]]; then 168 | ping $inet_addr_to_test 64 3 > /dev/null 169 | else 170 | ping -c 3 $inet_addr_to_test > /dev/null 171 | fi 172 | if [ $? != 0 ]; then 173 | echo "This script needs internet to resolve MX records." 174 | exit 1 175 | fi 176 | } 177 | 178 | function usage { 179 | echo "$PROGRAM $PROGRAM_VERSION $PROGRAM_BUILD" 180 | echo "$AUTHOR" 181 | echo "$CONTACT" 182 | echo "" 183 | echo "Usage: ./emailCheck.sh /path/to/emailList" 184 | echo "Email list needs to be a list of one email per line, encoded in UTF8 Unix format." 185 | echo "Checks if emails are RFC822 valid, corrects known typos in domain names, checks for valid MX records and checks against known ambiguous mail adresses." 186 | echo "" 187 | echo "Outputs 4 files with suffixes:" 188 | echo "$VALID_PREFIX: All emails that seem valid." 189 | echo "$NON_RFC_COMPLIANT_PREFIX: All emails that aren't RFC822 compliant." 190 | echo "$MISSING_MX_PREFIX: All emails of which domain doesn't have valid MX records." 191 | echo "$AMBIGUOUS_PREFIX: All emails which seem ambiguous." 192 | exit 1 193 | } 194 | 195 | function loop { 196 | local input="${1}" 197 | local output_rfc_non_compliant="${2}" 198 | local output_missing_mx="${3}" 199 | local output_tmp="${4}" 200 | 201 | echo "Checking emails." 202 | 203 | count=0 204 | while IFS=$CSV_INPUT_DELIMITER read $CSV_READ; do 205 | 206 | email=$(lowercase "$email") 207 | checkRFC822 "$email" 208 | if [ $? -eq 1 ]; then 209 | INCORRECT_MAILS=$((INCORRECT_MAILS+1)) 210 | echo "$email" >> "$output_rfc_non_compliant" 211 | continue 212 | fi 213 | 214 | newemail=$(checkDomains "$email") 215 | 216 | ## Ugly hack because incorrect_domains can't be increased directly in function checkDomains 217 | if [ "$newemail" != "$email" ]; then 218 | INCORRECT_DOMAINS=$((INCORRECT_DOMAINS+1)) 219 | email="$newemail" 220 | fi 221 | 222 | checkMXDomains "$email" 223 | if [ $? -eq 1 ]; then 224 | INCORRECT_MX=$((INCORRECT_MX+1)) 225 | echo "$email" >> "$output_missing_mx" 226 | continue 227 | fi 228 | 229 | eval "echo \"$CSV_WRITE\" >> \"$output_tmp\"" 230 | count=$((count+1)) 231 | if [ $((count % 1000)) -eq 0 ]; then 232 | echo "Time: $SECONDS - $count email addresses processed so far." 233 | fi 234 | done <"$input" 235 | } 236 | 237 | function sortAmbiguous { 238 | local input="${1}" 239 | local output_ambiguous="${2}" 240 | local output_valid="${3}" 241 | 242 | # Test for username and domain 243 | if [ $CSV_EMAIL_IS_FIRST_COLUMN == false ]; then 244 | BEGIN=$CSV_INPUT_DELIMITER 245 | else 246 | BEGIN='^' 247 | fi 248 | 249 | cmd="$BEGIN""test@|""$BEGIN""example@|""$BEGIN""exemple@|""$BEGIN""spam@|""$BEGIN""noreply@|""$BEGIN""no-reply@|@test\.|@example\.|@exemple\.|@spam\." 250 | eval 'egrep $cmd < "$input" > "$output_ambiguous"' 251 | eval 'egrep -v $cmd < "$input" > "$output_valid"' 252 | 253 | AMBIGUOUS_MAILS=$(wc -l < "$output_ambiguous") 254 | VALID_MAILS=$(wc -l < "$output_valid") 255 | } 256 | 257 | checkEnvironment 258 | 259 | if ([ "$1" == "" ] || [ ! -f "$1" ]) ; then 260 | usage 261 | fi 262 | 263 | input="$1" 264 | input_path="$(dirname $1)" 265 | input_file="$(basename $1)" 266 | output_tmp="$input_path/$TMP_PREFIX.$input_file" 267 | output_valid="$input_path/$VALID_PREFIX.$input_file" 268 | output_missing_mx="$input_path/$MISSING_MX_PREFIX.$input_file" 269 | output_non_rfc_compliant="$input_path/$NON_RFC_COMPLIANT_PREFIX.$input_file" 270 | output_ambiguous="$input_path/$AMBIGUOUS_PREFIX.$input_file" 271 | 272 | if [ -f "$output_tmp" ]; then 273 | rm -f "$output_tmp" 274 | fi 275 | if [ -f "$output_valid" ]; then 276 | rm -f "$output_valid" 277 | fi 278 | if [ -f "$output_missing_mx" ]; then 279 | rm -f "$output_missing_mx" 280 | fi 281 | if [ -f "$output_non_rfc_compliant" ]; then 282 | rm -f "$output_non_rfc_compliant" 283 | fi 284 | if [ -f "$output_ambiguous" ]; then 285 | rm -f "$output_ambiguous" 286 | fi 287 | 288 | loop "$input" "$output_non_rfc_compliant" "$output_missing_mx" "$output_tmp" 289 | if [ ! -f "$output_tmp" ]; then 290 | echo "No valid emails found. Check if your file has only email addresses, or configure the read process accordingly to read a multicolumn CSV file in source header." 291 | echo "Also, if your file comes from Windows, convert it using dos2unix first." 292 | else 293 | sortAmbiguous "$output_tmp" "$output_ambiguous" "$output_valid" 294 | fi 295 | 296 | echo "" 297 | echo "$INCORRECT_MAILS non rfc822 compliant emails are in [$output_non_rfc_compliant]." 298 | echo "$INCORRECT_DOMAINS emails had incorrect domains and have been corrected." 299 | echo "$INCORRECT_MX emails are missing mx records in their domain in [$output_missing_mx]." 300 | echo "$AMBIGUOUS_MAILS are ambiguous emails in [$output_ambiguous]." 301 | echo "$VALID_MAILS emails seem valid in [$output_valid]." 302 | -------------------------------------------------------------------------------- /repairbadblocks.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | PROGRAM="repairbadsectors" 4 | AUTHOR="Orsiris de Jong & Frederick Fouquet" 5 | VERSION=0.1 6 | PROGRAM_BUILD=2017041101 7 | 8 | function Usage { 9 | echo "Repair bad blocks from hard disk" 10 | echo "CAUTION: Use at your own risk, repair attempts are non destructive but a mechanic disk may totally fail under stress." 11 | echo "Always make nice backups first !" 12 | echo "" 13 | echo "This script has 3 different bad block correction functions:" 14 | echo "dd intelligent repair:" 15 | echo " Makes a list of all bad sectors with [badblock], then read the sectors to a tmp file, zero fill them, and write the data back from the tmp file." 16 | echo "" 17 | echo "hdparm repair:" 18 | echo " Makes a list of all bad sectors with [badblock], then reads them with hdparm, and tries to write them on read failure." 19 | echo "" 20 | echo "dd dumb repair:" 21 | echo " Same as dd intelligent repair, but instead of reading a list of bad sectors, proceeds for all sectors or a sector span." 22 | echo "" 23 | echo "Procedure explanation:" 24 | echo "" 25 | echo "Whenever a block of 4KB (8x512B sectors) is written to a disk, the disk firmware reallocates the sector if the sector is marked bad." 26 | echo "Warning: both dd intelligent repair and hdparm repair may not find bad sectors to repair if they are in \"current pending\" SMART status" 27 | echo "as the badblock utility won't see them." 28 | echo "dd dumb repair will find them, but will take painfully long to cover the whole disk." 29 | echo "dd dumb repair should be used to cover areas of the disk." 30 | echo "" 31 | echo "Usage:" 32 | echo "./repairbadblocks.sh /dev/sdX /tmp/directory" 33 | echo "/dev/sdX is the disk you want to repair" 34 | echo "/tmp/directory is a temporary directory containing sector data and bad blocks lists. Make sure it is a ramdrive or another disk than the one your're trying to repair." 35 | exit 1 36 | } 37 | 38 | #Todo real menu 39 | #Todo spinner 40 | 41 | function isNumeric { 42 | eval "local value=\"${1}\"" # Needed so variable variables can be processed 43 | 44 | local re="^-?[0-9]+([.][0-9]+)?$" 45 | if [[ $value =~ $re ]]; then 46 | echo 1 47 | else 48 | echo 0 49 | fi 50 | } 51 | 52 | function VerComp { 53 | if [ "$1" == "" ] || [ "$2" == "" ]; then 54 | echo "Bogus Vercomp values [$1] and [$2]." 55 | return 1 56 | fi 57 | 58 | if [[ $1 == $2 ]] 59 | then 60 | echo 0 61 | return 62 | fi 63 | 64 | local IFS=. 65 | local i ver1=($1) ver2=($2) 66 | # fill empty fields in ver1 with zeros 67 | for ((i=${#ver1[@]}; i<${#ver2[@]}; i++)) 68 | do 69 | ver1[i]=0 70 | done 71 | for ((i=0; i<${#ver1[@]}; i++)) 72 | do 73 | if [[ -z ${ver2[i]} ]] 74 | then 75 | # fill empty fields in ver2 with zeros 76 | ver2[i]=0 77 | fi 78 | if ((10#${ver1[i]} > 10#${ver2[i]})) 79 | then 80 | echo 1 81 | return 82 | fi 83 | if ((10#${ver1[i]} < 10#${ver2[i]})) 84 | then 85 | echo 2 86 | return 87 | fi 88 | done 89 | 90 | echo 0 91 | return 92 | } 93 | 94 | function readBadBlocks { 95 | local drive="${1}" 96 | local tmp_dir="${2}" 97 | 98 | echo "Reading bad blocks from $drive" 99 | badblocks -b 4096 -c 8 -sv -o "$tmp_dir/badblocks.$(basename $drive)" $drive 100 | } 101 | 102 | #Todo function readWriteNonDestructiveBadBlocks 103 | #function readWriteDestructiveBadBlocks 104 | 105 | function hdparmRepair { 106 | local drive="${1}" 107 | local tmp_dir="${2}" 108 | local lbablock="${3}" 109 | 110 | local badblockscount 111 | local counter=0 112 | local block 113 | local sector_begin 114 | local sector_end 115 | local result 116 | 117 | # Todo, what output does badblocks provide ? 118 | # Todo specific lbablock or badblocks 119 | readBadBlocks "$drive" "$tmp_dir" 120 | badblockscount=$(wc -l < "$tmp_dir/badblocks.$(basename $drive)") 121 | 122 | while read $block; do 123 | echo "Reading sector $block ($counter / $badblockscount)" 124 | sector_begin=$((block*8)) 125 | sector_end=$((sector_begin+7)) 126 | for sector in $(seq $sector_begin $sector_end); do 127 | hdparm --read-sector $sector $drive > /dev/null 128 | result=$? 129 | if [ $result -eq 0 ]; then 130 | echo "Sector [$sector] seems okay." 131 | elif [ $result -eq 5 ]; then 132 | echo "Sector [$sector] seems bad. Trying to rewrite it with zeros." 133 | hdparm --write-sector $sector --yes-i-know-what-i-am-doing $drive 134 | elif [ $result -eq 19 ]; then 135 | echo "Missing disk [$drive]." 136 | exit 2 137 | elif [ $result -eq 25 ]; then 138 | echo "Guru meditation failure with guru code [$result]." 139 | exit 3 140 | fi 141 | done 142 | counter=$((counter+1)) 143 | done < "$tmp_dir/badblocks.$(basename $drive)" 144 | echo "Finished repairs. Please do a smart long test on drive [$drive]." 145 | exit 0 146 | } 147 | 148 | function ddIntelligentRepair { 149 | local drive="${1}" 150 | local tmp_dir="${2}" 151 | local lbablock="${3}" 152 | 153 | local badblockscount 154 | # TODO lbablock or readBadBlocks 155 | readBadBlocks "$drive" "$tmp_dir" 156 | badblockscount=$(wc -l < "$tmp_dir/badblocks.$(basename $drive)") 157 | 158 | while read $block; do 159 | echo "Trying to repair block [$block] ($counter / $badblockscount)." 160 | sector=$((block*8)) 161 | dd if=$drive iflag=direct of="$tmp_dir/badblock.$sector.$(basename $drive)" bs=4096 count=1 skip=$sector > /dev/null 162 | dd if=/dev/zero of=$drive oflag=direct bs=4096 count=1 skip=$sector > /dev/null 163 | dd if="$tmp_dir/badblock.$block.$(basename $drive)" of=$drive oflag=direct bs=4096 count=1 skip=$sector > /dev/null 164 | if [ $? == 0 ]; then 165 | rm -f "$tmp_dir/badblock.$block.$(basename $drive)" 166 | else 167 | echo "Failed to dd write block [$block]." 168 | fi 169 | counter=$((counter+1)) 170 | done < "$tmp_dir/badblocks.$(basename $drive)" 171 | echo "Finished repairs. Please do a smart long test on drive [$drive]." 172 | exit 0 173 | } 174 | 175 | 176 | function ddDumbRepair { 177 | local drive="${1}" 178 | local tmp_dir="${2}" 179 | 180 | local block=0 181 | local read_result=0 182 | local continue=true 183 | 184 | local begin_block=0 185 | local end_block=0 186 | 187 | read -r -p "Beginning block number (0) ? " begin_block 188 | read -r -p "Ending block number (end of disk) ?" end_block 189 | 190 | if [ $(isNumeric "$begin_block") -eq 1 ]; then 191 | block=$begin_block 192 | fi 193 | 194 | if [ $(isNumeric "$end_block") -eq 0 ]; then 195 | end_block=0 196 | fi 197 | 198 | while [ $continue == true ]; do 199 | dd if=$drive iflag=direct of="$tmp_dir/badblock.tmp.$(basename $drive)" bs=4096 count=1 skip=$block > /dev/null 2>&1 200 | read_result=$? 201 | dd if=/dev/zero of=$drive oflag=direct bs=4096 count=1 skip=$block > /dev/null 2>&1 202 | dd if="$tmp_dir/badblock.tmp.$(basename $drive)" of=$drive oflag=direct bs=4096 count=1 skip=$block > /dev/null 2>&1 203 | block=$((block+1)) 204 | if [ $((block % 1000)) -eq 0 ]; then 205 | echo "Processed [$block] blocks." 206 | fi 207 | if [ $end_block -ne 0 ] && [ $block -gt $end_block ]; then 208 | continue=false 209 | elif [ $end_block -eq 0 ] && [ $read_result -ne 0 ]; then 210 | continue=false 211 | fi 212 | done 213 | exit 0 214 | } 215 | 216 | function confirmation { 217 | read -r -p "Are you sure to proceed (yes/NO) ?" ack 218 | if [ "$ack" == "yes" ] || [ "$ack" == "YES" ]; then 219 | return 1 220 | else 221 | return 0 222 | fi 223 | } 224 | 225 | function checkEnvironnment { 226 | 227 | if type dd > /dev/null 2>&1; then 228 | DD_PRESENT=true 229 | else 230 | echo "[dd] not found, will not provide dd repair options." 231 | DD_PRESENT=false 232 | fi 233 | 234 | if type badblocks > /dev/null 2>&1; then 235 | BADBLOCKS_PRESENT=true 236 | else 237 | echo "[badblocks] not found, repair time will be *much* longer." 238 | BADBLOCKS_PRESENT=false 239 | fi 240 | 241 | if type hdparm > /dev/null 2>&1; then 242 | hdparm_ver=$(hdparm -V | cut -f2 -d'v') 243 | if [ $(VerComp "$hdparm_ver" "8.0") -lt 2 ]; then 244 | HDPARM_PRESENT=true 245 | else 246 | echo "[hdparm] needs to be >= v8.0 to support repairs. Will not provide hdparm repair option." 247 | HDPARM_PRESENT=false 248 | fi 249 | else 250 | echo "[hdparm] not found, will not provide hdparm repair option." 251 | fi 252 | 253 | if ([ $DD_PRESENT == false ] && ([ $BADBLOCKS_PRESENT == false ] || [ $HDPARM_PRESENT == false ])); then 254 | echo "No required repair tools found. Cannot continue." 255 | exit 1 256 | fi 257 | } 258 | 259 | if [ "$1" != "" ] && [ "$2" != "" ]; then 260 | if [ ! -e "$1" ] || [ ! -w "$2" ]; then 261 | Usage 262 | else 263 | DRIVE="$1" 264 | TMP_DIR="$2" 265 | fi 266 | else 267 | Usage 268 | fi 269 | 270 | checkEnvironnment 271 | 272 | if ([ $DD_PRESENT == true ] && [ $BADBLOCKS_PRESENT == true ]); then 273 | echo "Launch intelligent dd repair" 274 | confirmation 275 | if [ $? -eq 1 ]; then 276 | ddIntelligentRepair "$DRIVE" "$TMP_DIR" 277 | fi 278 | fi 279 | 280 | if ([ $HDPARM_PRESENT == true ] && [ $BADBLOCKS_PRESENT == true ]); then 281 | echo "Launch hdparm repair" 282 | confirmation 283 | if [ $? -eq 1 ]; then 284 | hdparmRepair "$DRIVE" "$TMP_DIR" 285 | fi 286 | fi 287 | 288 | if [ $DD_PRESENT == true ]; then 289 | echo "Launch dumb dd repair (painfully long)" 290 | confirmation 291 | if [ $? -eq 1 ]; then 292 | ddDumbRepair "$DRIVE" "$TMP_DIR" 293 | fi 294 | fi 295 | 296 | echo "No option selected." 297 | exit 0 298 | 299 | -------------------------------------------------------------------------------- /ssh_jail.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | PROGRAM="ssh_jail.sh" # Basic ssh shell jail creation script 4 | AUTHOR="(L) 2014 by Orsiris \"Ozy\" de Jong" 5 | CONTACT="http://www.netpower.fr - ozy@netpower.fr" 6 | PROGRAM_BUILD=2108201403 7 | 8 | ## Creates a SSH chroot jail where a given user can login and execute a minimal set of programs 9 | ## Binaries specified by BINARIES_TO_COPY will be available 10 | ## You may also specifiy /usr/bin and /lib in EXT_DIRS to gain multiple binaries 11 | 12 | ## If you need support for editing programs, consider adding /usr/share/terminfo (CentOS) or /lib/terminfo directories (Debian) to EXT_DIRS 13 | 14 | 15 | # List of binary files to copy to chrooted environment. All dependencies will also be copied. 16 | BINARIES_TO_COPY="/usr/bin/chmod;/usr/bin/chown;/usr/bin/ls;/usr/bin/cat;/usr/bin/ln;/usr/bin/cp;/usr/sbin/ldconfig" # Works on CentOS 7 17 | #BINARIES_TO_COPY="/bin/chmod;/bin/chown;/bin/ls;/bin/cat;/bin/ln;/bin/cp;/bin/mv;/bin/rm;/usr/bin/curl" # Works on Debian 6 18 | 19 | # Directories to copy to chrooted environment 20 | # /etc/php.d is needed to support php modules 21 | # /usr/share/terminfo or /lib/terminfo is needed to support interactive programs like nano or php 22 | # /usr/share/snmp is needed for php-snmp module 23 | # /usr/share/zoneinfo is needed for php-composer 24 | EXT_DIRS="/usr/bin;/usr/lib64;/usr/share/php;/usr/share/terminfo;/usr/include;/etc/php.d;/usr/share/snmp;/usr/share/zoneinfo;/etc/ld.so.conf.d" # Works on CentOS 7 (replace lib by lib64 if needed) 25 | #EXT_DIRS="/bin;/lib;/lib/terminfo" # Works on Debian 6 (replace lib by lib64 if needed) 26 | #EXT_DIRS="/lib/terminfo" 27 | 28 | # Empty directories to create in chrooted environment types (lib and lib64 directories are already included) 29 | DIRS_TO_CREATE="/dev;/etc;/var/tmp;/tmp" 30 | 31 | # Additional files to copy to chrooted environment 32 | # /etc/resolv.conf is needed to enable internet access 33 | # /etc/pki/tls/certs/ca-bundle.crt and /etc/pki/tls/certs/ca-bundle.trust.crt are needed to enable SSL certificate verification 34 | FILES_TO_COPY="/etc/localtime;/etc/passwd;/etc/group;/etc/resolv.conf;/etc/pki/tls/certs/ca-bundle.crt;/etc/pki/tls/certs/ca-bundle.trust.crt;/etc/ld.so.conf" 35 | 36 | # Default group for chrooted users 37 | #GROUP=chroot 38 | GROUP=apache 39 | 40 | # Shell to use in chrooted environment 41 | SHELL=/usr/bin/bash 42 | 43 | # ADD chroot entry to ssh server 44 | ADD_SSH_CHROOT_ENTRY=yes 45 | 46 | # Use alternative chroot method without SSH (login chroot script) 47 | USE_CHROOT_SCRIPT=no 48 | 49 | ## END OF STANDARD CONFIGURATION OPTIONS ########################################################### 50 | 51 | CHROOT_SCRIPT_PATH=/bin/chrootusers 52 | USER_HOME=/home 53 | 54 | # Basic directories present on most linux flavors 55 | BASIC_DIRS="/usr/bin;/usr/lib;/usr/lib64" 56 | # Create symlinks for BASIC_DIRS to root (used on most linux flavors) 57 | CREATE_SYMLINKS="yes" 58 | 59 | SSHD_CONFIG="/etc/ssh/sshd_config" 60 | 61 | # Prevent cp -i alias that stays interactive 62 | alias cp=cp 63 | 64 | function LogDebug 65 | { 66 | if [ "$DEBUG" == "yes" ] 67 | then 68 | echo "$1" 69 | fi 70 | } 71 | 72 | function CheckErr 73 | { 74 | if [ $? != 0 ] 75 | then 76 | exec_error=1 77 | echo "Failed on task: $1" 78 | fi 79 | } 80 | 81 | function Usage 82 | { 83 | echo "$PROGRAM build $PROGRAM_BUILD" 84 | echo $AUTHOR 85 | echo $CONTACT 86 | echo "" 87 | echo "$0 [--alt-chroot-dir=/jail/dir] [--add-program=/path/to/program] [-f]" 88 | echo "Creates user and sets shell jail to its home directory" 89 | echo "--add-program=/some/program Adds a program and its dependencies to an existing chrooted user jail" 90 | echo "--alt-chroot-dir=/another/home By default, chroot is created in /home/. You can specify an alternate root here and chroot will be created in /" 91 | echo " This parameter also triggers a new chrootscript with the alternative home" 92 | echo "-f Forces deletion of existing user and chroot scripts" 93 | exit 128 94 | } 95 | 96 | function CheckEnvironment 97 | { 98 | case $(uname -m) in 99 | 100 | x86_64) 101 | ARCH=x64 102 | LIB="/lib64;/lib" 103 | ;; 104 | i686) 105 | ARCH=x86 106 | LIB=/lib 107 | ;; 108 | *) 109 | ARCH=unknown 110 | ;; 111 | 112 | esac 113 | 114 | if [ "$ARCH" == "unknown" ] 115 | then 116 | echo "Unknown architecture" 117 | exit 1 118 | fi 119 | 120 | if ! type -p bash > /dev/null 2>&1 121 | then 122 | echo "Cannot find bash shell environment." 123 | exit 1 124 | fi 125 | 126 | if ! type -p env > /dev/null 2>&1 127 | then 128 | echo "Cannot find env." 129 | exit 1 130 | else 131 | ENV_BINARY=$(type -p env) 132 | fi 133 | 134 | if ! type -p chroot > /dev/null 2>&1 135 | then 136 | echo "Cannot find chroot executable." 137 | exit 1 138 | else 139 | CHROOT_BINARY=$(type -p chroot) 140 | CHROOT_BINARY_ALT="$CHROOT_BINARY""_alt" 141 | fi 142 | 143 | if ! type -p ldd > /dev/null 2>&1 144 | then 145 | echo "Cannot find ldd executable." 146 | exit 1 147 | fi 148 | 149 | if ! type -p ldconfig > /dev/null 2>&1 150 | then 151 | echo "Cannot find ldconfig executable." 152 | exit 1 153 | fi 154 | 155 | if ! type -p ln > /dev/null 2>&1 156 | then 157 | echo "Cannot find ln executable." 158 | exit 1 159 | else 160 | LN_BINARY=$(type -p ln) 161 | fi 162 | 163 | } 164 | 165 | function AddUserAndGroup 166 | { 167 | echo "Creating group $GROUP" 168 | groupadd "$GROUP" 169 | 170 | echo "Creating user $LOGIN" 171 | # Returns 1 if user doesn't exist 172 | id -nu "$LOGIN" > /dev/null 2>&1 173 | if [ $? != 0 ] || [ "$force" == "1" ] 174 | then 175 | if [ "$USE_CHROOT_SCRIPT" == "yes" ] 176 | then 177 | chroot="-s $CHROOT_SCRIPT_PATH" 178 | else 179 | chroot="" 180 | fi 181 | useradd -c "User chrooted" -d "$CHROOT_DIR/" -g "$GROUP" $chroot "$LOGIN" 182 | CheckErr "Adding user $LOGIN" 183 | else 184 | echo "User $LOGIN already exists." 185 | exit 1 186 | fi 187 | 188 | echo "Please enter password for $LOGIN" 189 | passwd "$LOGIN" > /dev/null 190 | CheckErr "Creating password for $LOGIN" 191 | if ! [ -d "$CHROOT_DIR" ] 192 | then 193 | echo "Creating home directory" 194 | mkdir -p "$CHROOT_DIR/" 195 | CheckErr "Create directory $CHROOT_DIR/" 196 | fi 197 | } 198 | 199 | function AddBinaryPaths 200 | { 201 | OLD_IFS=$IFS 202 | IFS=";" 203 | for binary in $BINARIES_TO_COPY 204 | do 205 | DIRS_TO_CREATE="$DIRS_TO_CREATE;$(dirname $binary)" 206 | done 207 | IFS=$OLD_IFS 208 | LogDebug "List of directories to create: $DIRS_TO_CREATE" 209 | } 210 | 211 | function CreatePaths 212 | { 213 | OLD_IFS=$IFS 214 | IFS=";" 215 | LogDebug "$DIRS_TO_CREATE" 216 | for dir in $DIRS_TO_CREATE 217 | do 218 | if ! [ -d "$CHROOT_DIR$dir" ] 219 | then 220 | LogDebug "Creating $CHROOT_DIR$dir" 221 | mkdir -p "$CHROOT_DIR$dir" 222 | CheckErr "Create directory $CHROOT_DIR$dir" 223 | fi 224 | 225 | chmod 700 "$CHROOT_DIR$dir" 226 | CheckErr "command chmod 700 $CHROOT_DIR$dir" 227 | done 228 | IFS=$OLD_IFS 229 | } 230 | 231 | # Adds binaries and dependancy libs to chroot (list separated by semicolons given as argument) 232 | function AddBinaries 233 | { 234 | OLD_IFS=$IFS 235 | IFS=";" 236 | for binary in $1 237 | do 238 | dependencies="" 239 | if [ ! -d "$CHROOT_DIR$(dirname $binary)" ] 240 | then 241 | mkdir -p "$CHROOT_DIR$(dirname $binary)" 242 | CheckErr "Creating $CHROOT_DIR$(dirname $binary)" 243 | fi 244 | 245 | LogDebug "Copying $binary to $CHROOT_DIR$binary" 246 | cp $binary "$CHROOT_DIR$binary" 247 | CheckErr "Copy $binary to $CHROOT_DIR$binary" 248 | 249 | IFS=$OLD_IFS 250 | # Get all dependant libraries from binary (ldd sometimes gives output at first column and sometimes at third depending on the type of dependency) 251 | dependencies=$(ldd $binary | awk '{print $1}' | grep "^/") 252 | dependencies="$dependencies"$'\n'$(ldd $binary | awk '{print $3}' | grep "^/") 253 | for dependency in $dependencies 254 | do 255 | dependency_dir=$(dirname $dependency) 256 | if [ ! -d "$CHROOT_DIR$dependency_dir" ] 257 | then 258 | mkdir -p "$CHROOT_DIR$dependency_dir" 259 | CheckErr "Creating $CHROOT_DIR$dependency_dir" 260 | fi 261 | 262 | if [ ! -f "$CHROOT_DIR$dependency" ] 263 | then 264 | LogDebug "Copying dependency $dependency to $CHROOT_DIR$dependency_dir/" 265 | cp "$dependency" "$CHROOT_DIR$dependency_dir/" 266 | CheckErr "Copy $dependency to $CHROOT_DIR$dependency_dir/" 267 | fi 268 | done 269 | IFS=";" 270 | done 271 | IFS=$OLD_IFS 272 | } 273 | 274 | function CopyFiles 275 | { 276 | OLD_IFS=$IFS 277 | IFS=";" 278 | for file in $FILES_TO_COPY 279 | do 280 | LogDebug "Copying $file to $CHROOT_DIR$(dirname $file)/" 281 | if ! [ -d "$CHROOT_DIR$(dirname $file)" ] 282 | then 283 | mkdir -p "$CHROOT_DIR$(dirname $file)" 284 | CheckErr "reate $CHROOT_DIR$(dirname $file)" 285 | fi 286 | cp "$file" "$CHROOT_DIR$(dirname $file)/" 287 | CheckErr "Copy $file to $CHROOT_DIR$(dirname $file)/" 288 | done 289 | IFS=$OLD_IFS 290 | } 291 | 292 | function CopyDirs 293 | { 294 | OLD_IFS=$IFS 295 | IFS=";" 296 | for dir in $EXT_DIRS 297 | do 298 | mkdir -p "$CHROOT_DIR$dir" 299 | CheckErr "Creating $CHROOT_DIR$dir" 300 | LogDebug "Copying directory $dir to $CHROOT_DIR/" 301 | cp -R "$dir/" "$CHROOT_DIR$(dirname $dir)" 302 | CheckErr "Copy $dir to $CHROOT_DIR/" 303 | done 304 | IFS=$OLD_IFS 305 | } 306 | 307 | function AllowChrootBinary 308 | { 309 | if [ -f "$CHROOT_BINARY" ] 310 | then 311 | cp "$CHROOT_BINARY" "$CHROOT_BINARY_ALT" 312 | CheckErr "Copy $CHROOT_BINARY to $CHROOT_BINARY_ALT" 313 | # Setuid for allowing execution of chroot binary by normal user 314 | chmod 4755 "$CHROOT_BINARY_ALT" 315 | CheckErr "chmod 4755 $CHROOT_BINARY_ALT" 316 | fi 317 | } 318 | 319 | function AddSSHChrootEntry 320 | { 321 | if [ -f $SSHD_CONFIG ] 322 | then 323 | LogDebug "Adding chroot entry to $SSHD_CONFIG" 324 | echo "" >> $SSHD_CONFIG 325 | echo "Match User $LOGIN" >> $SSHD_CONFIG 326 | echo " ChrootDirectory $CHROOT_DIR" >> $SSHD_CONFIG 327 | echo " AllowTCPForwarding no" >> $SSHD_CONFIG 328 | echo " X11Forwarding no" >> $SSHD_CONFIG 329 | CheckErr "Adding chroot entry to $SSHD_CONFIG" 330 | 331 | echo "Don't forget to reload sshd." 332 | else 333 | echo "Cannot find $SSHD_CONFIG path" 334 | exit 1 335 | fi 336 | } 337 | 338 | function CreateChrootScript 339 | { 340 | if ! [ -f "$CHROOT_SCRIPT_PATH" ] || [ "$force_script" == "1" ] 341 | then 342 | echo "Creating $CHROOT_SCRIPT_PATH" 343 | cat > "$CHROOT_SCRIPT_PATH" << EXTSCRIPT 344 | #!/bin/bash 345 | if [ -d "$USER_HOME/\$USER" ] 346 | then 347 | exec -c "$CHROOT_BINARY_ALT" "$USER_HOME/\$USER" "$ENV_BINARY" -i TERM="\$TERM" HOME="/" $SHELL --login -i 348 | else 349 | echo "No home directory" 350 | exit 1 351 | fi 352 | EXTSCRIPT 353 | CheckErr "Create script $CHROOT_SCRIPT_PATH" 354 | chmod 555 "$CHROOT_SCRIPT_PATH" 355 | CheckErr "chmod 555 $CHROOT_SCRIPT_PATH" 356 | else 357 | echo "$CHROOT_SCRIPT_PATH already exists. Use -f to override." 358 | fi 359 | } 360 | 361 | function AddSymlinks 362 | { 363 | # Add symlinks from BASIC_DIRS to root dir 364 | OLD_IFS=$IFS 365 | IFS=";" 366 | 367 | for link in $BASIC_DIRS 368 | do 369 | LogDebug "Creating symlink $link -> /$(basename $link)" 370 | $LN_BINARY -s "$link" "$CHROOT_DIR/$(basename $link)" 371 | CheckErr "Creating $link symlink" 372 | done 373 | IFS=$OLD_IFS 374 | } 375 | 376 | function AddSpecialFiles 377 | { 378 | if ! [ -c "$CHROOT_DIR/dev/null" ] 379 | then 380 | mknod "$CHROOT_DIR/dev/null" c 1 3 -m 666 381 | CheckErr "Creating /dev/null in jail" 382 | fi 383 | 384 | if ! [ -c "$CHROOT_DIR/dev/console" ] 385 | then 386 | mknod "$CHROOT_DIR/dev/console" c 5 1 -m 622 387 | CheckErr "Creating /dev/console in jail" 388 | fi 389 | 390 | if ! [ -c "$CHROOT_DIR/dev/zero" ] 391 | then 392 | mknod "$CHROOT_DIR/dev/zero" c 1 5 -m 666 393 | CheckErr "Creating /dev/zero in jail" 394 | fi 395 | 396 | if ! [ -c "$CHROOT_DIR/dev/ptmx" ] 397 | then 398 | mknod "$CHROOT_DIR/dev/ptmx" c 5 2 -m 666 399 | CheckErr "Creating /dev/ptmx in jail" 400 | fi 401 | 402 | if ! [ -c "$CHROOT_DIR/dev/tty" ] 403 | then 404 | mknod "$CHROOT_DIR/dev/tty" c 5 1 -m 666 405 | CheckErr "Creating /dev/tty in jail" 406 | fi 407 | 408 | if ! [ -c "$CHROOT_DIR/dev/random" ] 409 | then 410 | mknod "$CHROOT_DIR/dev/random" c 1 8 -m 444 411 | CheckErr "Creating /dev/random in jail" 412 | fi 413 | 414 | if ! [ -c "$CHROOT_DIR/dev/urandom" ] 415 | then 416 | mknod "$CHROOT_DIR/dev/urandom" c 1 9 -m 444 417 | CheckErr "Creating /dev/urandom in jail" 418 | fi 419 | 420 | chown root:tty $CHOOT_DIR/dev/{console,ptmx,tty} 421 | CheckErr "Taking ownership of /dev/console /dev/ptmx and /dev/tty" 422 | } 423 | 424 | function Runldconfig 425 | { 426 | LogDebug "Running ldconfig in chrooted environment" 427 | chroot $CHROOT_DIR /usr/sbin/ldconfig 428 | CheckErr "ldconfig failed" 429 | } 430 | 431 | function SetPermissions 432 | { 433 | LogDebug "Changing owner of $CHROOT_DIR to $LOGIN:$GROUP" 434 | chown -R "$LOGIN:$GROUP" "$CHROOT_DIR/" 435 | CheckErr "chown -R $LOGIN:$GROUP $CHROOT_DIR/" 436 | 437 | chown "root:root" "$CHROOT_DIR" 438 | CheckErr "chown root:root $CHROOT_DIR" 439 | 440 | chmod 755 "$CHROOT_DIR" 441 | CheckErr "chmod 755 $CHROOT_DIR" 442 | } 443 | 444 | if [ "$1" == "" ] 445 | then 446 | Usage 447 | else 448 | LOGIN="$1" 449 | fi 450 | 451 | CheckEnvironment 452 | 453 | force=0 454 | force_script=0 455 | for i in "$@" 456 | do 457 | case $i in 458 | --add-program=*) 459 | ADD_PROGRAM="${i##*=}" 460 | ;; 461 | --alt-chroot-dir=*) 462 | USER_HOME="${i##*=}" 463 | force_script=1 464 | ;; 465 | -f) 466 | force=1 467 | force_script=1 468 | ;; 469 | esac 470 | done 471 | CHROOT_DIR="$USER_HOME/$LOGIN" 472 | 473 | # Add arch dependend lib path to directory list 474 | DIRS_TO_CREATE="$DIRS_TO_CREATE;$CHROOT_DIR" 475 | 476 | if ! [ "$ADD_PROGRAM" == "" ] 477 | then 478 | AddBinaryPaths 479 | CreatePaths 480 | AddBinaries "$ADD_PROGRAM" 481 | SetPermissions 482 | exit 483 | else 484 | # Normal program run 485 | AddUserAndGroup 486 | CreatePaths 487 | CopyDirs 488 | CopyFiles 489 | if [ "$CREATE_SYMLINKS" == "yes" ] 490 | then 491 | AddSymlinks 492 | fi 493 | AddBinaries "$SHELL;$ENV_BINARY;$BINARIES_TO_COPY" 494 | if [ "$ADD_SSH_CHROOT_ENTRY" == "yes" ] 495 | then 496 | AddSSHChrootEntry 497 | fi 498 | if [ "$USE_CHROOT_SCRIPT" == "yes" ] 499 | then 500 | AllowChrootBinary 501 | CreateChrootScript 502 | fi 503 | AddSpecialFiles 504 | Runldconfig 505 | SetPermissions 506 | fi 507 | 508 | if [ "$exec_error" == "1" ] 509 | then 510 | echo "Script finished with errors for user $LOGIN" 511 | else 512 | echo "Created chrooted user $LOGIN" 513 | fi 514 | -------------------------------------------------------------------------------- /virsh/mk_vm.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Machine create script 2025040401 4 | 5 | # TODO: Since libvirt 9.1.0, q35 vm include itco watchdog by default, se we should remove i6300esb by default as per https://libvirt.org/formatdomain.html#watchdog-devices 6 | # TODO: add --boot uefi as option 7 | # OS (get with osinfo-query os) 8 | OS_VARIANT=rhel9.5 9 | ISO=/data/public_vm/ISO/AlmaLinux-9.5-x86_64-dvd.iso 10 | ISO=/opt/AlmaLinux-9.5-x86_64-dvd.iso 11 | #OS_VARIANT=debian12 12 | #ISO=/data/public_vm/ISO/debian-12.9.0-amd64-DVD-1.iso 13 | #OS_VARIANT=win2k22 14 | #ISO=/data/public_vm/ISO/fr-fr_windows_server_2022_x64_dvd_9f7d1adb.iso 15 | #OS_VARIANT=opensuse15.5 16 | #ISO=/data/public_vm/ISO/grommunio.x86_64-latest.install.iso 17 | #OS_VARIANT=debian12 18 | #ISO=/data/public_vm/ISO/proxmox-mail-gateway_8.1-1.iso 19 | #OS_VARIANT=freebsd14.0 20 | #ISO=/opt/OPNsense-25.1-dvd-amd64.iso 21 | 22 | # BOOT_TYPE = cdrom when no kernel can be directly loaded (used for appliances and windows) 23 | #BOOT_TYPE=cdrom 24 | 25 | TENANT=npf 26 | VM=___vmname___.${TENANT}.local 27 | DISKSIZE=100G 28 | DISKPATH=/data/public/${TENANT} 29 | #DISKPATH=/var/lib/libvirt/images 30 | DISKFULLPATH="${DISKPATH}/${VM}-disk0.qcow2" 31 | VCPUS=4 32 | RAM=4096 33 | 34 | # IO MODE io_uring is fastest on io intesive VMs 35 | # IO MODE native with threads is fast 36 | # IO MODE native has good latency 37 | IO_MODE=,io="native" 38 | # For IO intensive machines, the followng will improve latency at the cost of slighty lower IOPS 39 | # io=threads still reduces performances overall, so io=native,iothread=x is good 40 | #IO_MODE=,io="native,driver.iothread=1,driver.queues=${VCPUS} --iothreads 1" 41 | #IO_MODE=,io="io_uring,driver.queues=${VCPUS} --iothreads 4" 42 | 43 | # Paramètres VM 44 | PRODUCT=vm_elconf 45 | VERSION=5.0 46 | MANUFACTURER=NetPerfect 47 | VENDOR=netperfect_vm 48 | 49 | #IP= 50 | #NETMASK= 51 | #GATEWAY= 52 | #NAMESERVER= 53 | 54 | NPF_TARGET=generic 55 | #NPF_USER_NAME=user 56 | #NPF_USER_PASSWORD= 57 | #NPF_ROOT_PASSWORD= 58 | 59 | # Host names can only contain the characters 'a-z', 'A-Z', '0-9', '-', or '.', cannot start or end with '-' 60 | NPF_HOSTNAME="${VM}" 61 | if [ "${IP}" != "" ] && [ "${NETMASK}" != "" ]; then 62 | # This is for pre script to pick up 63 | NPF_NETWORK="${IP}:${NETMASK}:${GATEWAY}:${NAMESERVER}" 64 | # This is for anaconda installer to pick up 65 | IP="ip=${IP}::${GATEWAY}:${NETMASK}:${VM}:none nameserver=${NAMESERVER}" 66 | #IP="ip=192.168.151.11::192.168.151.254:255.255.255.0:${VM}:none nameserver=192.168.151.254" 67 | fi 68 | 69 | #VIDEO="--graphics none" 70 | VIDEO="--video virtio --graphics vnc,listen=127.0.0.1,keymap=fr" 71 | 72 | #BRIDGE="--network bridge=br_dmzint" 73 | #BRIDGE="--network bridge=br_${TENANT}" 74 | #BRIDGE="--network bridge=br_cloudstack" 75 | BRIDGE="--network bridge=br_net0" 76 | #PCI_PASSTHROUGH="--host-device pci_0000_03_00_0 --network none" 77 | #BRIDGE="--network bridge=br_dmzext" 78 | 79 | INST="inst.text inst.lang=en_US inst.keymap=fr" 80 | KICKSTART=/root/ks.rhel9.cfg 81 | 82 | ## Prepare commands 83 | if [ ${OS_VARIANT:0:3} == "win" ] || [ "$BOOT_TYPE" == "cdrom" ]; then 84 | BOOT_ARGS="--cdrom ${ISO}" 85 | extra_args="" 86 | else 87 | BOOT_ARGS="--location ${ISO}" 88 | extra_args="console=tty0 console=ttyS0,115200n8 ${INST} ${IP}" 89 | fi 90 | 91 | if [ "${KICKSTART}" != "" ]; then 92 | extra_args="${extra_args} inst.ks=file:/$(basename ${KICKSTART}) inst.nosave=all_ks" 93 | KICKSTART_INJECT="--initrd-inject ${KICKSTART}" 94 | fi 95 | 96 | [ -n "${NPF_TARGET}" ] && extra_args="${extra_args} NPF_TARGET=${NPF_TARGET}" 97 | [ -n "${NPF_USER_NAME}" ] && extra_args="${extra_args} NPF_USER_NAME=${NPF_USER_NAME}" 98 | [ -n "${NPF_USER_PASSWORD}" ] && extra_args="${extra_args} NPF_USER_PASSWORD=${NPF_USER_PASSW0RD}" 99 | [ -n "${NPF_ROOT_PASSWORD}" ] && extra_args="${extra_args} NPF_ROOT_PASSWORD=${NPF_ROOT_PASSWORD}" 100 | [ -n "${NPF_HOSTNAME}" ] && extra_args="${extra_args} NPF_HOSTNAME=${NPF_HOSTNAME}" 101 | [ -n "${NPF_NETWORK}" ] && extra_args="${extra_args} NPF_NETWORK=${NPF_NETWORK}" 102 | 103 | ## Create tenant dir if not exit 104 | [ ! -d "$DISKPATH" ] && mkdir "$DISKPATH" && chown qemu:qemu "$DISKPATH" 105 | 106 | # -o cluster_size=64k 64k is optimal for DB environment (and is default value), should match underlying storage cluster size (recordsize on zfs) 107 | # -o lazy_refcounts: less IO (we mark image as dirty and it will be counted later). DO NOT ENABLE THIS since it may corrupt images and require a repair after a power loss 108 | # -o refcount_bits= : 16 bits as default, 64 bits is default, the more the faster, but will need more memory cache to be configured 109 | disk_cmd="qemu-img create -f qcow2 -o extended_l2=on -o preallocation=metadata -o cluster_size=64k "${DISKFULLPATH}" ${DISKSIZE}" 110 | echo $disk_cmd 111 | $disk_cmd 112 | if [ $? != 0 ]; then 113 | echo "Disk creation failed" 114 | exit 1 115 | fi 116 | 117 | if [ ${OS_VARIANT:0:3} == "win" ] || [ "$BOOT_TYPE" == "cdrom" ]; then 118 | vm_cmd='virt-install --name '${VM}' --ram '${RAM}' --vcpus '${VCPUS}' --cpu host --os-variant '${OS_VARIANT}' --disk path='${DISKFULLPATH}',bus=virtio,cache=none'${IO_MODE}' --channel unix,mode=bind,target_type=virtio,name=org.qemu.guest_agent.0 --watchdog i6300esb,action=reset --sound none --boot hd --autostart --sysinfo smbios,bios.vendor='${VENDOR}',system.manufacturer='${MANUFACTURER}',system.version='${VERSION}',system.product='${PRODUCT}' '${BOOT_ARGS}' '${VIDEO}' '${BRIDGE}' --autoconsole text' 119 | else 120 | vm_cmd='virt-install --name '${VM}' --ram '${RAM}' --vcpus '${VCPUS}' --cpu host --os-variant '${OS_VARIANT}' --disk path='${DISKFULLPATH}',bus=virtio,cache=none'${IO_MODE}' --channel unix,mode=bind,target_type=virtio,name=org.qemu.guest_agent.0 --watchdog i6300esb,action=reset --sound none --boot hd --autostart --sysinfo smbios,bios.vendor='${VENDOR}',system.manufacturer='${MANUFACTURER}',system.version='${VERSION}',system.product='${PRODUCT}' '${BOOT_ARGS}' --extra-args "'${extra_args}'" '${KICKSTART_INJECT}' '${VIDEO}' '${BRIDGE}' --autoconsole text' 121 | fi 122 | echo $vm_cmd 123 | eval "$vm_cmd" 124 | -------------------------------------------------------------------------------- /virsh/vm_move.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Quick and dirty KVM VM local storage move script 4 | # Written by Orsiris de Jong 5 | # Usage 6 | # ./vm_move.sh vm_name destination_path [dryrun=true|false] 7 | SCRIPT_BUILD=2025042402 8 | 9 | # SCRIPT ARGUMENTS 10 | VM_NAME="${1:-false}" 11 | DST_DIR="${2:-false}" 12 | DRY_RUN="${3:-false}" 13 | DELETE_SOURCE="${4:-false}" 14 | 15 | 16 | LOG_FILE="/var/log/$(basename $0).log" 17 | 18 | 19 | SCRIPT_GOOD=true 20 | 21 | log() { 22 | __log_line="${1}" 23 | __log_level="${2:-INFO}" 24 | 25 | __log_line="${__log_level}: ${__log_line}" 26 | echo "${__log_line}" >> "${LOG_FILE}" 27 | echo "${__log_line}" 28 | 29 | if [ "${__log_level}" = "ERROR" ]; then 30 | SCRIPT_GOOD=false 31 | fi 32 | } 33 | 34 | 35 | move_storage() { 36 | # Filter disk images only 37 | xml_dumped=false 38 | xml_ok=false 39 | disk_pivoted=false 40 | vm_xml="${DST_DIR}/${VM_NAME}.inactive.$(date +"%Y%m%dT%H%M%S").xml" 41 | for disk in $(virsh domblklist "$VM_NAME" --details | grep "file" | grep "disk" | awk '{print $3"="$4}'); do 42 | disk_name="$(echo "${disk}" | awk -F'=' '{print $1}')" 43 | src_disk_path="$(echo "${disk}" | awk -F'=' '{print $2}')" 44 | if [ ! -f "${src_disk_path}" ]; then 45 | log "Source disk ${disk_name} not found in ${src_disk_path}" "ERROR" 46 | break 47 | fi 48 | dst_disk_path="${DST_DIR}/$(basename "${src_disk_path}")" 49 | log "Found disk ${disk_name} in ${src_disk_path}" 50 | if [ "${xml_dumped}" == false ]; then 51 | log "Exporting ${VM_NAME} to ${vm_xml}" 52 | virsh dumpxml --inactive "${VM_NAME}" > "${vm_xml}" 53 | if [ $? != 0 ]; then 54 | log "VM $VM_NAME dump failed. Not trying to migrate it" "ERROR" 55 | break 56 | else 57 | xml_dumped=true 58 | xml_ok=true 59 | fi 60 | log "Undefining $vm_name" 61 | [ "${DRY_RUN}" == true ] || virsh undefine "${VM_NAME}" 62 | if [ $? != 0 ]; then 63 | log "Undefining $VM_NAME failed" "ERROR" 64 | break 65 | fi 66 | fi 67 | 68 | if [ "${src_disk_path}" == "${dst_disk_path}" ]; then 69 | log "Source and destination are identical. Won't do anything" "ERROR" 70 | continue 71 | fi 72 | 73 | log "Moving disk ${disk_name} to ${dst_disk_path}" 74 | [ "${DRY_RUN}" == true ] || virsh blockcopy "${VM_NAME}" "${disk_name}" --dest="${dst_disk_path}" --wait --pivot --verbose 75 | if [ $? != 0 ]; then 76 | log "Failed to blockcopy $VM_NAME to $DST_DIR/$vm_disk" "ERROR" 77 | disk_pivoted=false 78 | break 79 | else 80 | # Check if disk image is not in use anymore 81 | lsof "${src_disk_path}" > /dev/null 2>&1 82 | if [ $? -eq 0 ]; then 83 | if [ "${DRY_RUN}" == true ]; then 84 | log "Disk ${src_disk_path} is in use by $(lsof "${src_disk_path}")" 85 | else 86 | log "Disk ${src_disk_path} is still in use by $(lsof "${src_disk_path}")" "ERROR" 87 | fi 88 | else 89 | disk_pivoted=true 90 | if [ "${DELETE_SOURCE}" == true ]; then 91 | log "Deleting source disk ${src_disk_path}" 92 | rm "${src_disk_path}" || log "Cannot delete old disk image" "ERROR" 93 | else 94 | old_disk_path="${src_disk_path}.old.$(date +"%Y%m%dT%H%M%S")" 95 | log "Renaming original file to ${old_disk_path}" 96 | mv "${src_disk_path}" "${old_disk_path}" || log "Cannot rename old disk image" "ERROR" 97 | fi 98 | fi 99 | fi 100 | log "Modifying disk path from \"${src_disk_path}\" to \"${dst_disk_path}\"" 101 | sed -i "s#${src_disk_path}#${dst_disk_path}#g" "${vm_xml}" 102 | if [ $? != 0 ]; then 103 | log "Failed to modify XML file $vm_vml" "ERROR" 104 | if [ "${disk_pivoted}" == false ]; then 105 | log "Stopping operation since disks did not pivot yet" 106 | break 107 | else 108 | log "Continuing operations, but xml file is bad" "ERROR" 109 | xml_ok=false 110 | fi 111 | fi 112 | if ! grep "${dst_disk_path}" "$vm_xml" > /dev/null 2>&1; then 113 | log "XML file check did not succeed" "ERROR" 114 | xml_ok=false 115 | fi 116 | done 117 | 118 | if [ "${xml_ok}" == true ]; then 119 | log "Defining VM ${VM_NAME} from ${vm_xml}" 120 | [ "${DRY_RUN}" == true ] || virsh define "$vm_xml" 121 | if [ $? != 0 ]; then 122 | log "Failed to redefine ${VM_NAME}" "ERROR" 123 | fi 124 | else 125 | log "XML file is not okay, cannot redefine ${VM_NAME} from ${vm_xml}" "ERROR" 126 | log "VM ${VM_NAME} is in transient state. Please repair." "ERROR" 127 | fi 128 | } 129 | 130 | 131 | [ "${DRY_RUN}" == true ] && log "Running in DRY mode. Nothing will actually be done" "NOTICE" 132 | 133 | if [ "${VM_NAME}" == false ] || [ "${DST_DIR}" == false ]; then 134 | log "Please run $0 vm_name dest_dir [dry_run] [delete_source]" 135 | exit 1 136 | fi 137 | 138 | DST_DIR="$(realpath "${DST_DIR}")" 139 | 140 | [ ! -d "${DST_DIR}" ] && mkdir "${DST_DIR}" 141 | if [ ! -w "${DST_DIR}" ]; then 142 | log "Destination dir ${DST_DIR} is not writable" "ERROR" 143 | exit 1 144 | fi 145 | 146 | if ! virsh list --name | grep "^${VM_NAME}$" > /dev/null 2>&1; then 147 | log "VM ${VM_NAME} not found via virsh list" "ERROR" 148 | exit 1 149 | fi 150 | 151 | move_storage "${VM_NAME}" "${DST_DIR}" 152 | 153 | 154 | log "List of transient domains" 155 | virsh list --transient 156 | virsh list --transient >> "$LOG_FILE" 2>&1 157 | 158 | log "End of line" 159 | 160 | if [ "${SCRIPT_GOOD}" == true ]; then 161 | exit 0 162 | else 163 | exit 1 164 | fi 165 | -------------------------------------------------------------------------------- /virsh/vm_snapshots.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | vms=$(virsh list --all --name) 4 | for vm in ${vms[@]}; do 5 | echo "VM Name: $vm" 6 | virsh snapshot-list --tree $vm 7 | done 8 | -------------------------------------------------------------------------------- /virsh/vm_stats.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | printf '%-20s %-4s %-8s\n' "Vm Name" "CPU(s)" "RAM (MiB)" 4 | 5 | mapfile -t vm_array < <( virsh list --all --name ) 6 | 7 | while read -r vm 8 | do 9 | if [ ! -z "$vm" ]; then 10 | MEM_USED=$(($(virsh dominfo $vm | grep "Max memory" | cut -f 7 -d " ") / 1024)) 11 | CPU_USED=$(virsh dominfo $vm | grep "CPU(s)" | cut -f 10 -d " ") 12 | printf "%-40s %4s | %8s\n" "$vm" "$CPU_USED" "$MEM_USED" 13 | MEM_SUM=$((MEM_SUM + MEM_USED)) 14 | CPU_SUM=$((CPU_SUM + CPU_USED)) 15 | fi 16 | done < <( printf '%s\n' "${vm_array[@]}") 17 | 18 | printf -- '-%.0s' {1..36} 19 | printf '\n%-40s %4s | %8s\n' "Totals:" "$CPU_SUM" "$MEM_SUM" 20 | --------------------------------------------------------------------------------