├── .gitignore ├── IP_Search.cmd ├── 7za ├── x64 │ ├── 7za.dll │ ├── 7za.exe │ └── 7zxa.dll ├── x86 │ ├── 7za.dll │ ├── 7za.exe │ └── 7zxa.dll ├── license.txt ├── history.txt └── readme.txt ├── wget ├── x64 │ └── wget.exe ├── x86 │ └── wget.exe ├── COPYING └── README.md ├── Data ├── .gitignore └── README.md ├── LICENSE ├── Data_Update.cmd ├── Ping_Test.cmd ├── Network_Calc.cmd ├── README.md └── Script_Calc.cmd /.gitignore: -------------------------------------------------------------------------------- 1 | # MaxMind License Key 2 | license_key.txt 3 | -------------------------------------------------------------------------------- /IP_Search.cmd: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/IP_Search.cmd -------------------------------------------------------------------------------- /7za/x64/7za.dll: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/7za/x64/7za.dll -------------------------------------------------------------------------------- /7za/x64/7za.exe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/7za/x64/7za.exe -------------------------------------------------------------------------------- /7za/x64/7zxa.dll: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/7za/x64/7zxa.dll -------------------------------------------------------------------------------- /7za/x86/7za.dll: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/7za/x86/7za.dll -------------------------------------------------------------------------------- /7za/x86/7za.exe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/7za/x86/7za.exe -------------------------------------------------------------------------------- /7za/x86/7zxa.dll: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/7za/x86/7zxa.dll -------------------------------------------------------------------------------- /wget/x64/wget.exe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/wget/x64/wget.exe -------------------------------------------------------------------------------- /wget/x86/wget.exe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ScriptTiger/IP/HEAD/wget/x86/wget.exe -------------------------------------------------------------------------------- /Data/.gitignore: -------------------------------------------------------------------------------- 1 | # Data files 2 | exit-addresses 3 | reputation.data 4 | ip-block-list 5 | .wget-hsts 6 | *.csv 7 | *.txt 8 | -------------------------------------------------------------------------------- /Data/README.md: -------------------------------------------------------------------------------- 1 | # IP Data Files 2 | All the GeoLite2 CSV and Tor exit-addresses files should be placed here, with no subdirectories beyond this point. 3 | 4 | The GeoLite2 databases are distributed under the Creative Commons Attribution-ShareAlike 4.0 International License. 5 | 6 | These scripts reference using GeoLite2 data created by MaxMind, available from: 7 | http://www.maxmind.com -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2025 ScriptTiger 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /7za/license.txt: -------------------------------------------------------------------------------- 1 | 7-Zip Extra 2 | ~~~~~~~~~~~ 3 | License for use and distribution 4 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 5 | 6 | Copyright (C) 1999-2019 Igor Pavlov. 7 | 8 | 7-Zip Extra files are under the GNU LGPL license. 9 | 10 | 11 | Notes: 12 | You can use 7-Zip Extra on any computer, including a computer in a commercial 13 | organization. You don't need to register or pay for 7-Zip. 14 | 15 | 16 | GNU LGPL information 17 | -------------------- 18 | 19 | This library is free software; you can redistribute it and/or 20 | modify it under the terms of the GNU Lesser General Public 21 | License as published by the Free Software Foundation; either 22 | version 2.1 of the License, or (at your option) any later version. 23 | 24 | This library is distributed in the hope that it will be useful, 25 | but WITHOUT ANY WARRANTY; without even the implied warranty of 26 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 27 | Lesser General Public License for more details. 28 | 29 | You can receive a copy of the GNU Lesser General Public License from 30 | http://www.gnu.org/ 31 | 32 | -------------------------------------------------------------------------------- /Data_Update.cmd: -------------------------------------------------------------------------------- 1 | @echo off 2 | 3 | rem ===== 4 | rem For more information on ScriptTiger and more ScriptTiger scripts visit the following URL: 5 | rem https://scripttiger.github.io/ 6 | rem Or visit the following URL for the latest information on this ScriptTiger script: 7 | rem https://github.com/ScriptTiger/IP 8 | rem ===== 9 | 10 | set LKT=%~dps0license_key.txt 11 | 12 | if exist %LKT% ( 13 | set /p MML=<%LKT% 14 | ) else ( 15 | echo You currently don't have a MaxMind license key configured. 16 | echo Please visit the following address to obtain a free or paid key: 17 | echo https://www.maxmind.com/en/my_license_key 18 | set /p MML=MaxMind License Key: 19 | ) 20 | 21 | if not exist %LKT% echo %MML%>%LKT% 22 | 23 | set MM=https://download.maxmind.com/app/geoip_download?suffix=zip 24 | set CURL="%MM%&license_key=%MML%&edition_id=GeoLite2-City-CSV" 25 | set AURL="%MM%&license_key=%MML%&edition_id=GeoLite2-ASN-CSV" 26 | set TURL=https://check.torproject.org/exit-addresses 27 | set AVURL=http://reputation.alienvault.com/reputation.data 28 | set IPBURL=https://www.snort.org/downloads/ip-block-list 29 | set DATA=%~dps0Data 30 | set TOR=%DATA%\exit-addresses 31 | set IN=%DATA%\geoip_download*-CSV 32 | set AV=%DATA%\reputation.data 33 | set IPB=ip-block-list 34 | set IPBF=%DATA%\%IPB% 35 | 36 | if "%PROCESSOR_ARCHITECTURE%"=="" (set ARCH=x86) else (set ARCH=%PROCESSOR_ARCHITECTURE:~-2%) 37 | 38 | set ZA=%~dps07za\x%ARCH%\7za.exe 39 | set WGET="%~dps0wget\x%ARCH%\wget.exe" -nv --show-progress -P "%DATA%" 40 | 41 | echo Downloading archives... 42 | 43 | :Download_City 44 | timeout /t 15 /nobreak > nul 45 | %WGET% %CURL% || goto Download_City 46 | 47 | :Download_ASN 48 | timeout /t 15 /nobreak > nul 49 | %WGET% %AURL% || goto Download_ASN 50 | 51 | echo Deleting old Tor list... 52 | 53 | if exist "%TOR%" del "%TOR%" 54 | 55 | echo Downloading new Tor list... 56 | 57 | :Download_Tor 58 | timeout /t 15 /nobreak > nul 59 | %WGET% %TURL% || goto Download_Tor 60 | 61 | echo Deleting old AlienVault data... 62 | 63 | if exist "%AV%" del "%AV%" 64 | 65 | echo Downloading new AlienVault data... 66 | 67 | :Download_AV 68 | timeout /t 15 /nobreak > nul 69 | %WGET% %AVURL% || goto Download_AV 70 | 71 | echo Deleting old Snort IP Blacklist... 72 | 73 | if exist "%IPBF%" del "%IPBF%" 74 | 75 | echo Downloading new Snort IP Blacklist... 76 | 77 | :Download_IPB 78 | timeout /t 15 /nobreak > nul 79 | %WGET% %IPBURL% || goto Download_IPB 80 | 81 | echo Converting Snort IP Blacklist to CRLF... 82 | 83 | if exist "%IPBF%.tmp" del "%IPBF%.tmp" 84 | 85 | ( 86 | for /f %%0 in ('type "%IPBF%"') do echo %%0 87 | ) > "%IPBF%.tmp" 88 | 89 | del "%IPBF%" 90 | 91 | ren "%IPBF%.tmp" "%IPB%" 92 | 93 | echo Extracting archives... 94 | 95 | %ZA% e -O%DATA% -y %IN% > nul 96 | 97 | echo Deleting archives... 98 | 99 | del %IN% 100 | 101 | echo Update complete 102 | if not "%1"=="/q" pause 103 | -------------------------------------------------------------------------------- /Ping_Test.cmd: -------------------------------------------------------------------------------- 1 | @echo off 2 | 3 | rem ===== 4 | rem For more information on ScriptTiger and more ScriptTiger scripts visit the following URL: 5 | rem https://scripttiger.github.io/ 6 | rem Or visit the following URL for the latest information on this ScriptTiger script: 7 | rem https://github.com/ScriptTiger/IP 8 | rem ===== 9 | 10 | echo Initializing... 11 | 12 | rem ===== 13 | rem Environmental setup 14 | rem ===== 15 | 16 | setlocal ENABLEDELAYEDEXPANSION 17 | 18 | rem set ADAPTER variable here to your primary network adapter (ie: Ethernet 2, Wireless Network Connection 6, etc.) 19 | set ADAPTER= 20 | 21 | if "%ADAPTER%"=="" ( 22 | echo Please set the ADAPTER variable in the script before continuing. 23 | pause 24 | exit 25 | ) 26 | set Count=1 27 | for /f "tokens=3" %%0 in ( 28 | 'netsh interface ip show config "%ADAPTER%" ^| findstr "IP Address:"' 29 | ) do set IP=%%0 30 | for /f "tokens=3" %%0 in ( 31 | 'netsh interface ip show config "%ADAPTER%" ^| findstr "Default Gateway:"' 32 | ) do set GATEWAY=%%0 33 | 34 | rem ===== 35 | rem Initialize and count indices 36 | rem ===== 37 | 38 | set Index=0 39 | for /f "tokens=1,2" %%0 in ( 40 | 'echo L0 127.0.0.1^ 41 | & echo L0 localhost^ 42 | & echo Eth0 !IP!^ 43 | & echo Gateway !GATEWAY!^ 44 | & echo Google google.com' 45 | ) do ( 46 | set /a Index=!Index!+1 47 | set Index_!Index!_=%%0,%%1,0,0,0,0 ,0 48 | ) 49 | 50 | rem ===== 51 | rem Save current time 52 | rem ===== 53 | 54 | set Time=%date% @ %time% 55 | 56 | rem ===== 57 | rem Begin main loop 58 | rem ===== 59 | 60 | for /l %%0 in () do ( 61 | 62 | rem ===== 63 | rem Recurse indices 64 | rem 1.) name, 2.) ip/url, 3.) success, 4.) average ms, 5.) percent, 6.) display, 7.) ms/* 65 | rem ===== 66 | 67 | for /f "tokens=1,2,3,4,5,7 delims==," %%a in ( 68 | 'set index_' 69 | ) do ( 70 | set Success=%%d 71 | set AverageMS=%%e 72 | 73 | rem ===== 74 | rem Time display on unsuccessful ping 75 | rem ===== 76 | 77 | set MSDisplay=%%f 78 | set TimeMS=* 79 | 80 | rem ===== 81 | rem Ping host 82 | rem ===== 83 | 84 | for /f "tokens=1,2,4,8,10 delims== nul 136 | ) -------------------------------------------------------------------------------- /7za/history.txt: -------------------------------------------------------------------------------- 1 | 7-Zip Extra history 2 | ------------------- 3 | 4 | This file contains only information about changes related to that package exclusively. 5 | The full history of changes is listed in history.txt in main 7-Zip program. 6 | 7 | 8 | 19.00 2019-02-21 9 | ------------------------- 10 | - Encryption strength for 7z archives was increased: 11 | the size of random initialization vector was increased from 64-bit to 128-bit, 12 | and the pseudo-random number generator was improved. 13 | - Some bugs were fixed. 14 | 15 | 16 | 18.06 2018-12-30 17 | ------------------------- 18 | - The speed for LZMA/LZMA2 compressing was increased by 3-10%, 19 | and there are minor changes in compression ratio. 20 | - Some bugs were fixed. 21 | 22 | 23 | 18.05 2018-04-30 24 | ------------------------- 25 | - The speed for LZMA/LZMA2 compressing was increased 26 | by 8% for fastest/fast compression levels and 27 | by 3% for normal/maximum compression levels. 28 | 29 | 30 | 18.03 beta 2018-03-04 31 | ------------------------- 32 | - The speed for single-thread LZMA/LZMA2 decoding 33 | was increased by 30% in x64 version and by 3% in x86 version. 34 | - 7-Zip now can use multi-threading for 7z/LZMA2 decoding, 35 | if there are multiple independent data chunks in LZMA2 stream. 36 | 37 | 38 | 9.35 beta 2014-12-07 39 | ------------------------------ 40 | - SFX modules were moved to LZMA SDK package. 41 | 42 | 43 | 9.34 alpha 2014-06-22 44 | ------------------------------ 45 | - Minimum supported system now is Windows 2000 for EXE and DLL files. 46 | - all EXE and DLL files use msvcrt.dll. 47 | - 7zr.exe now support AES encryption. 48 | 49 | 50 | 9.18 2010-11-02 51 | ------------------------------ 52 | - New small SFX module for installers. 53 | 54 | 55 | 9.17 2010-10-04 56 | ------------------------------ 57 | - New 7-Zip plugin for FAR Manager x64. 58 | 59 | 60 | 9.10 2009-12-30 61 | ------------------------------ 62 | - 7-Zip for installers now supports LZMA2. 63 | 64 | 65 | 9.09 2009-12-12 66 | ------------------------------ 67 | - LZMA2 compression method support. 68 | - Some bugs were fixed. 69 | 70 | 71 | 4.65 2009-02-03 72 | ------------------------------ 73 | - Some bugs were fixed. 74 | 75 | 76 | 4.38 beta 2006-04-13 77 | ------------------------------ 78 | - SFX for installers now supports new properties in config file: 79 | Progress, Directory, ExecuteFile, ExecuteParameters. 80 | 81 | 82 | 4.34 beta 2006-02-27 83 | ------------------------------ 84 | - ISetProperties::SetProperties: 85 | it's possible to specify desirable number of CPU threads: 86 | PROPVARIANT: name=L"mt", vt = VT_UI4, ulVal = NumberOfThreads 87 | If "mt" is not defined, 7za.dll will check number of processors in system to set 88 | number of desirable threads. 89 | Now 7za.dll can use: 90 | 2 threads for LZMA compressing 91 | N threads for BZip2 compressing 92 | 4 threads for BZip2 decompressing 93 | Other codecs use only one thread. 94 | Note: 7za.dll can use additional "small" threads with low CPU load. 95 | - It's possible to call ISetProperties::SetProperties to specify "mt" property for decoder. 96 | 97 | 98 | 4.33 beta 2006-02-05 99 | ------------------------------ 100 | - Compressing speed and Memory requirements were increased. 101 | Default dictionary size was increased: Fastest: 64 KB, Fast: 1 MB, 102 | Normal: 4 MB, Max: 16 MB, Ultra: 64 MB. 103 | - 7z/LZMA now can use only these match finders: HC4, BT2, BT3, BT4 104 | 105 | 106 | 4.27 2005-09-21 107 | ------------------------------ 108 | - Some GUIDs/interfaces were changed. 109 | IStream.h: 110 | ISequentialInStream::Read now works as old ReadPart 111 | ISequentialOutStream::Write now works as old WritePart 112 | -------------------------------------------------------------------------------- /7za/readme.txt: -------------------------------------------------------------------------------- 1 | 7-Zip Extra 19.00 2 | ----------------- 3 | 4 | 7-Zip Extra is package of extra modules of 7-Zip. 5 | 6 | 7-Zip Copyright (C) 1999-2019 Igor Pavlov. 7 | 8 | 7-Zip is free software. Read License.txt for more information about license. 9 | 10 | Source code of binaries can be found at: 11 | http://www.7-zip.org/ 12 | 13 | This package contains the following files: 14 | 15 | 7za.exe - standalone console version of 7-Zip with reduced formats support. 16 | 7za.dll - library for working with 7z archives 17 | 7zxa.dll - library for extracting from 7z archives 18 | License.txt - license information 19 | readme.txt - this file 20 | 21 | Far\ - plugin for Far Manager 22 | x64\ - binaries for x64 23 | 24 | 25 | All 32-bit binaries can work in: 26 | Windows 2000 / 2003 / 2008 / XP / Vista / 7 / 8 / 10 27 | and in any Windows x64 version with WoW64 support. 28 | All x64 binaries can work in any Windows x64 version. 29 | 30 | All binaries use msvcrt.dll. 31 | 32 | 7za.exe 33 | ------- 34 | 35 | 7za.exe - is a standalone console version of 7-Zip with reduced formats support. 36 | 37 | Extra: 7za.exe : support for only some formats of 7-Zip. 38 | 7-Zip: 7z.exe with 7z.dll : support for all formats of 7-Zip. 39 | 40 | 7za.exe and 7z.exe from 7-Zip have same command line interface. 41 | 7za.exe doesn't use external DLL files. 42 | 43 | You can read Help File (7-zip.chm) from 7-Zip package for description 44 | of all commands and switches for 7za.exe and 7z.exe. 45 | 46 | 7za.exe features: 47 | 48 | - High compression ratio in 7z format 49 | - Supported formats: 50 | - Packing / unpacking: 7z, xz, ZIP, GZIP, BZIP2 and TAR 51 | - Unpacking only: Z, lzma, CAB. 52 | - Highest compression ratio for ZIP and GZIP formats. 53 | - Fast compression and decompression 54 | - Strong AES-256 encryption in 7z and ZIP formats. 55 | 56 | Note: LZMA SDK contains 7zr.exe - more reduced version of 7za.exe. 57 | But you can use 7zr.exe as "public domain" code. 58 | 59 | 60 | 61 | DLL files 62 | --------- 63 | 64 | 7za.dll and 7zxa.dll are reduced versions of 7z.dll from 7-Zip. 65 | 7za.dll and 7zxa.dll support only 7z format. 66 | Note: 7z.dll is main DLL file that works with all archive types in 7-Zip. 67 | 68 | 7za.dll and 7zxa.dll support the following decoding methods: 69 | - LZMA, LZMA2, PPMD, BCJ, BCJ2, COPY, 7zAES, BZip2, Deflate. 70 | 71 | 7za.dll also supports 7z encoding with the following encoding methods: 72 | - LZMA, LZMA2, PPMD, BCJ, BCJ2, COPY, 7zAES. 73 | 74 | 7za.dll and 7zxa.dll work via COM interfaces. 75 | But these DLLs don't use standard COM interfaces for objects creating. 76 | 77 | Look also example code that calls DLL functions (in source code of 7-Zip): 78 | 79 | 7zip\UI\Client7z 80 | 81 | Another example of binary that uses these interface is 7-Zip itself. 82 | The following binaries from 7-Zip use 7z.dll: 83 | - 7z.exe (console version) 84 | - 7zG.exe (GUI version) 85 | - 7zFM.exe (7-Zip File Manager) 86 | 87 | Note: The source code of LZMA SDK also contains the code for similar DLLs 88 | (DLLs without BZip2, Deflate support). And these files from LZMA SDK can be 89 | used as "public domain" code. If you use LZMA SDK files, you don't need to 90 | follow GNU LGPL rules, if you want to change the code. 91 | 92 | 93 | 94 | 95 | License FAQ 96 | ----------- 97 | 98 | Can I use the EXE or DLL files from 7-Zip in a commercial application? 99 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 100 | Yes, but you are required to specify in documentation for your application: 101 | (1) that you used parts of the 7-Zip program, 102 | (2) that 7-Zip is licensed under the GNU LGPL license and 103 | (3) you must give a link to www.7-zip.org, where the source code can be found. 104 | 105 | 106 | Can I use the source code of 7-Zip in a commercial application? 107 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 108 | Since 7-Zip is licensed under the GNU LGPL you must follow the rules of that license. 109 | In brief, it means that any LGPL'ed code must remain licensed under the LGPL. 110 | For instance, you can change the code from 7-Zip or write a wrapper for some 111 | code from 7-Zip and compile it into a DLL; but, the source code of that DLL 112 | (including your modifications / additions / wrapper) must be licensed under 113 | the LGPL or GPL. 114 | Any other code in your application can be licensed as you wish. This scheme allows 115 | users and developers to change LGPL'ed code and recompile that DLL. That is the 116 | idea of free software. Read more here: http://www.gnu.org/. 117 | 118 | 119 | 120 | Note: You can look also LZMA SDK, which is available under a more liberal license. 121 | 122 | 123 | --- 124 | End of document 125 | -------------------------------------------------------------------------------- /Network_Calc.cmd: -------------------------------------------------------------------------------- 1 | @echo off 2 | 3 | rem ===== 4 | rem For more information on ScriptTiger and more ScriptTiger scripts visit the following URL: 5 | rem https://scripttiger.github.io/ 6 | rem Or visit the following URL for the latest information on this ScriptTiger script: 7 | rem https://github.com/ScriptTiger/IP 8 | rem ===== 9 | 10 | rem ===== 11 | rem Parameters for Help 12 | rem ===== 13 | 14 | if /i "%~1"=="help" goto Help 15 | if /i "%~1"=="/help" goto Help 16 | if /i "%~1"=="-help" goto Help 17 | if /i "%~1"=="--help" goto Help 18 | if "%~1"=="?" goto Help 19 | if "%~1"=="/?" goto Help 20 | if "%~1"=="-?" goto Help 21 | if "%~1"=="--?" goto Help 22 | 23 | rem ===== 24 | rem Environmental setup 25 | rem ===== 26 | 27 | setlocal ENABLEDELAYEDEXPANSION 28 | 29 | set CALC=%~dp0Script_Calc.cmd 30 | 31 | rem ===== 32 | rem Convert to string using decimal 33 | rem ===== 34 | 35 | if "%1"=="/s" ( 36 | call :DecToStr DEC %2 %3 37 | echo !DEC! 38 | exit /b 39 | ) 40 | 41 | rem ===== 42 | rem Convert to decimal using string 43 | rem ===== 44 | 45 | if "%1"=="/d" ( 46 | call :StrToDec STR %2 %3 47 | if "!STR!"=="0x80000000" set STR=-2147483648 48 | if not "%2"=="/r" for /f %%0 in ('!CALC! /32bit !STR!') do set STR=%%0 49 | echo !STR! 50 | exit /b 51 | ) 52 | 53 | rem ===== 54 | rem Convert to network ID using IP/CIDR 55 | rem ===== 56 | 57 | if "%1"=="/id" ( 58 | for /f "tokens=1,2,3,4,5 delims=/" %%0 in ("%2") do ( 59 | call :CIDRToRMask %%1 60 | call :XOR MASK !RMASK! 61 | call :AND ID %%0 !MASK! 62 | echo !ID!/%%1 63 | ) 64 | exit /b 65 | ) 66 | 67 | rem ===== 68 | rem Interactive mode 69 | rem ===== 70 | 71 | if not "%1"=="" ( 72 | set IM=0 73 | goto Calc 74 | ) else set IM=1 75 | 76 | :Input 77 | set /p IP=IP address: || exit /b 78 | set /p MASK=Subnet Mask: || exit /b 79 | call :Calc !IP! !MASK! 80 | exit /b 81 | 82 | rem ===== 83 | rem Calculate network information based on IP and CIDR or mask 84 | rem ===== 85 | 86 | :Calc 87 | echo. 88 | if "%2"=="" set CIDR=1 89 | for /f "tokens=1,2 delims=/ " %%0 in ("%1 %2") do ( 90 | if "!CIDR!"=="1" ( 91 | set CIDR=%%1 92 | call :CIDRToRMask %%1 93 | call :XOR MASK !RMASK! 94 | call :AND ID %%0 !MASK! 95 | ) else ( 96 | set MASK=%%1 97 | call :XOR RMASK %%1 98 | call :MaskToCIDR %%1 99 | call :AND ID %%0 %%1 100 | ) 101 | call :OR BCAST %%0 !RMASK! 102 | call :StrToDec DIP %%0 103 | for /f %%a in ('!CALC! /32bit !DIP!') do set DIP=%%a 104 | call :StrToDec HOSTS !RMASK! 105 | if !HOSTS! leq 1 ( 106 | set /a HOSTS=!HOSTS!+1 107 | set START=!ID! 108 | set END=!BCAST! 109 | if !HOSTS!==2 echo Note: This point-to-point network does not contain a broadcast address.&echo This may cause problems for some devices. 110 | if !HOSTS!==1 echo Note: This is a unicast address and not a network. 111 | ) else ( 112 | set /a HOSTS=!HOSTS!-1 113 | call :StrToDec START !ID! 114 | set /a START=!START!+1 115 | call :DecToStr START !START! 116 | call :StrToDec END !BCAST! 117 | set /a END=!END!-1 118 | call :DecToStr END !END! 119 | ) 120 | if "!DIP!"=="0x80000000" set DIP=2147483648 121 | echo %%0 ^(!DIP!^) / !MASK! ^(!RMASK!^): 122 | echo !ID!/!CIDR! - !BCAST! 123 | echo !START! - !END! ^(!HOSTS! usable IPs^) 124 | ) 125 | if %IM%==1 pause 126 | exit /b 127 | 128 | rem ===== 129 | rem Dec<->Str conversions 130 | rem enable support for reverse (/r) IPs 131 | rem ===== 132 | 133 | :DecToStr 134 | if "%2"=="/r" shift 135 | if "%2"=="-2147483648" ( 136 | set /a SubStr1="128" 137 | set /a SubStr2="0" 138 | set /a SubStr3="0" 139 | set /a SubStr4="0" 140 | ) else ( 141 | set /a SubStr1="(%2>>24)&0xff" 142 | set /a SubStr2="(%2>>16)&0xff" 143 | set /a SubStr3="(%2>>8)&0xff" 144 | set /a SubStr4="%2&0xff" 145 | ) 146 | if "%1"=="/r" ( 147 | set %0=!SubStr4!.!SubStr3!.!SubStr2!.!SubStr1! 148 | ) else ( 149 | set %1=!SubStr1!.!SubStr2!.!SubStr3!.!SubStr4! 150 | ) 151 | exit /b 152 | 153 | :StrToDec 154 | if "%2"=="/r" shift 155 | call :StrToSubStr %2 156 | if "%1"=="/r" ( 157 | set /a %0="!SubStr1!|(!SubStr2!<<8)|(!SubStr3!<<16)|(!SubStr4!<<24)" 158 | ) else ( 159 | if "%2"=="128.0.0.0" ( 160 | set %1=0x80000000 161 | ) else ( 162 | set /a %1="(!SubStr1!<<24)|(!SubStr2!<<16)|(!SubStr3!<<8)|!SubStr4!" 163 | ) 164 | ) 165 | exit /b 166 | 167 | rem ===== 168 | rem CIDR<->(R)Mask conversions 169 | rem ===== 170 | 171 | :CIDRToRMask 172 | set RMASK=0 173 | set /a RCIDR=32-%1 174 | for /l %%a in (1,1,!RCIDR!) do set /a RMASK="(!RMASK!<<1)|1" 175 | call :DecToStr RMASK !RMASK! 176 | exit /b 177 | 178 | :MaskToCIDR 179 | call :StrToSubStr %1 180 | set CIDR=0 181 | for %%0 in (!SubStr1! !SubStr2! !SubStr3! !SubStr4!) do ( 182 | for /l %%a in (0,1,7) do ( 183 | set /a OCT="0x100-(1<<%%a)" 184 | if %%0==!OCT! set /a CIDR=!CIDR!+8-%%a 185 | ) 186 | ) 187 | exit /b 188 | 189 | rem ===== 190 | rem Convert an IP string to 4 octet substrings for individual calculation 191 | rem ===== 192 | 193 | :StrToSubStr 194 | for /f "tokens=1,2,3,4 delims=." %%0 in ("%1") do ( 195 | set SubStr1=%%0 196 | set SubStr2=%%1 197 | set SubStr3=%%2 198 | set SubStr4=%%3 199 | ) 200 | exit /b 201 | 202 | rem ===== 203 | rem Bitwise calculations 204 | rem ===== 205 | 206 | :AND 207 | :OR 208 | :XOR 209 | call :StrToDec Str %2 210 | if "%0"==":XOR" ( 211 | set /a Str="!Str!^^0xffffffff" 212 | ) else ( 213 | call :StrToDec Str2 %3 214 | if "%0"==":AND" (set /a Str="!Str!&!Str2!") else (set /a Str="!Str!|!Str2!") 215 | ) 216 | call :DecToStr %1 !Str! 217 | exit /b 218 | 219 | rem ===== 220 | rem Help 221 | rem ===== 222 | 223 | :Help 224 | echo /d [/r] ^ Retrieve decimal format using X.X.X.X format 225 | echo /s [/r] ^ Retrieve X.X.X.X format using decimal format 226 | echo /id ^ Retrieve network ID using X.X.X.X/CIDR format 227 | echo ^[/X / X.X.X.X] Retrieve network information 228 | exit /b -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Say Thanks!](https://img.shields.io/badge/Say%20Thanks-!-1EAEDB.svg)](https://docs.google.com/forms/d/e/1FAIpQLSfBEe5B_zo69OBk19l3hzvBmz3cOV6ol1ufjh0ER1q3-xd2Rg/viewform) 2 | 3 | **If you find these scripts useful, you may also be interested in checking out the reference implementation in the goIP repository, which contains much of the same functionality ported to Go and compiled to binaries for much faster results, as well as a REST API available for IP queries. 4 | goIP Reference Implementation: https://github.com/ScriptTiger/goIP/tree/main/ref 5 | goIP Binary Downloads: https://github.com/ScriptTiger/goIP/releases** 6 | 7 | **There is also a Java GUI client available as a front end for the REST API: 8 | GeoIP: https://github.com/ScriptTiger/GeoIP 9 | GeoIP Downloads: https://github.com/ScriptTiger/GeoIP/releases** 10 | 11 | # IP 12 | Tired of paying a subscription service to access an API to resolve geoIP data? Look no further! Resolving locally and maintaining your own database is easier than ever. This repo is everything IP: 32-bit/128-bit calculations, ping monitoring across multiple nodes, IPv4/IPv6 network calculations, geoip lookups. 13 | 14 | Using bitwise and otherwise calculations of numbers 32 bits and higher in a 32-bit environment to calculate 32-bit IPv4 and 128-bit IPv6 network information. 15 | 16 | 7-Zip and wget are also components to this project licensed separately in accordance with their attached documentation. 17 | 18 | Ping_Test: 19 | Monitor continuous ping statistics across a series of nodes. 20 | 21 | Script_Calc: 22 | Can calculate and convert numbers 32 bits and higher from a 32-bit environment. 23 | 24 | Network_Calc: 25 | Currently only calculates IPv4. Since IPv4 addresses take up exactly 32 bits, this script requires Script_Calc to accurately calculate the decimal conversion of an IP address because 32-bit environments are limited to only calculate 31-bit numbers because the 32nd bit is used for sign control in designating if a number is positive or negative. The decimal conversion is included in this script for the sole purpose of letting you know what it is, while all other calcluations are bitwise and don't require a decimal conversion. Once IPv6 is also included, Script_Calc will be required for all IPv6 calculations. 26 | Note: The reverse IP address decimal conversion (/d /r) intentionally stays within 32-bit limitations to be able to convert to various executable output that stores data as a reverse 32-bit number (i.e. NirSoft/Nir Sofer INI files). The forward decimal conversion is useful for various lookup tables that list networks in decimal (i.e. GeoLite2's optional decimal format). 27 | 28 | IP_Search: 29 | Currently only supports searching IPv4 addresses. Thanks to MaxMind, non-numerical data has multilingual support! You can also use this script both interactively and scripted. For scripted searches with pretty output, just send the language and IP to search for to the script like this: 30 | `IP_Search.cmd en 8.8.8.8` 31 | The above command will search for 8.8.8.8 (Google DNS) and output the results in English. 32 | For scripted searches in CSV format: 33 | `IP_Search.cmd csv 8.8.8.8 C:\SomeFile.csv` 34 | The above command will search for 8.8.8.8 (Google DNS) and output the results in CSV format to the file C:\SomeFile.csv. You can optionally not specify a file name to allow output to the console. 35 | Language options are as follows: 36 | de 37 | en 38 | es 39 | fr 40 | ja 41 | pt-BR 42 | ru 43 | zh-CN 44 | Because GeoLite2 is the free product from MaxMind, the accuracy is obviously not going to be as good as the paid product, and neither are 100%. The only thing this search script CAN do with nearly 100% accuracy, assuming you have just recently run the Data_Update.cmd, is detect Tor nodes, as the Tor exit node list is provided directly from the Tor Project and is separate from the MaxMind data. The following is the current output of the script: 45 | 46 | Output Field | Explanation 47 | ------------------------|------------------------------------------------------------------------------------------------------- 48 | ----- WAN Data ----- 49 | WAN: | Target IP's parent wide area network 50 | Continent: | Continent on which the target IP resides 51 | Country: | Country in which the target IP resides 52 | Subdivisin 1: | Major subdivision in which the target IP resides 53 | Subdivisin 2: | Minor subdivision in which the target IP resides 54 | City: | City in which the target IP resides 55 | Metro Code: | Metro code to the target IP's surrounding area 56 | Time Zone: | Time zone of the target IP's surrounding area 57 | EU: | Is the target IP in the EU (Yes/No) 58 | Registered Country: | Country to which the target IP is registered 59 | ------------------------| Displayed as: Country, Continent (Time zone) 60 | Represented Country: | Foreign national representation possessing the IP 61 | ------------------------| Displayed as: Country, Continent (Time zone) 62 | Known Proxy: | Does the target IP host a known proxy server (Yes/No) 63 | Post Code: | Post code to the target IP's surrounding area 64 | Google Maps: | Google Maps link to view the IP's approximate location 65 | Accuracy: | Approximate accuracy of the target IP's location results 66 | ----- ASN Data ----- 67 | ASN Network: | Network to which the target IP's covering ASN spans 68 | ASN: | Autonomous system number covering the target IP 69 | ISP: | ISP controlling the target IP's covering ASN 70 | ----- Other Data ----- 71 | Known Tor Exit: | Does the target IP host a known Tor exit node (Yes/No) 72 | Blacklisted by Snort: | Is the target IP blacklisted by Snort (Yes/No) 73 | 74 | Data_Update.cmd: 75 | Will automatically download the latest data from all sources to the Data subdirectory: 76 | GeoLite2 (requires free or paid MaxMind license key from https://www.maxmind.com/en/my_license_key written to license_key.txt in the same directory) 77 | https://check.torproject.org/exit-addresses 78 | http://reputation.alienvault.com/reputation.data 79 | https://talosintelligence.com/documents/ip-blacklist 80 | 81 | 82 | These scripts reference using GeoLite2 data created by MaxMind, available from: 83 | http://www.maxmind.com 84 | 85 | The GeoLite2 databases are distributed under the Creative Commons Attribution-ShareAlike 4.0 International License. 86 | 87 | You can download this repo from the below link to get started: 88 | https://github.com/ScriptTiger/IP/archive/master.zip 89 | 90 | # More About ScriptTiger 91 | 92 | For more ScriptTiger scripts and goodies, check out ScriptTiger's GitHub Pages website: 93 | https://scripttiger.github.io/ 94 | -------------------------------------------------------------------------------- /Script_Calc.cmd: -------------------------------------------------------------------------------- 1 | @echo off 2 | 3 | rem ===== 4 | rem For more information on ScriptTiger and more ScriptTiger scripts visit the following URL: 5 | rem https://scripttiger.github.io/ 6 | rem Or visit the following URL for the latest information on this ScriptTiger script: 7 | rem https://github.com/ScriptTiger/IP 8 | rem ===== 9 | 10 | setlocal ENABLEDELAYEDEXPANSION 11 | set bdat=%temp%\bits.dat 12 | set label=%1 13 | if not "%3"=="" goto !label:~1! 14 | if "!label:~,1!"=="/" set external=1&shift&goto !label:~1! 15 | 16 | rem The default operation of this script is to add two numbers 17 | rem Subtraction is thus treated as addition by adding a positive with a negative 18 | rem Script_Calc 1 1 results in 2 19 | rem Script_Calc 1 -1 results in 0 20 | :add 21 | call :detect_sign "%1" sign1 22 | call :detect_sign "%2" sign2 23 | call :remove_sign "%1" string1 24 | call :remove_sign "%2" string2 25 | set digit4=0 26 | set string3= 27 | if "!sign1!"=="!sign2!" ( 28 | call :add_calc 29 | ) else ( 30 | call :subtract_calc 31 | ) 32 | if "%3"=="" ( 33 | echo !string3! 34 | ) else ( 35 | set %3=!string3! 36 | ) 37 | exit /b 38 | 39 | rem Stacked bitwise function 40 | rem Converts all numbers to hexidecimal first to keep bit integrity 41 | rem Calculates 8 hexidecimal numbers at a time 42 | rem Outputs in hexidecimal 43 | :and 44 | :or 45 | :xor 46 | call :dec2hex "%2" bitwise1 47 | call :dec2hex "%3" bitwise2 48 | set bitwise1=!bitwise1:~2! 49 | set bitwise2=!bitwise2:~2! 50 | set bitwise4= 51 | :bin_loop 52 | call :shift_right "!bitwise1!" bitwise1 oct1 53 | call :shift_right "!bitwise2!" bitwise2 oct2 54 | if "%1"=="/and" set /a bitwise3="0x!oct1!&0x!oct2!" 55 | if "%1"=="/or" set /a bitwise3="0x!oct1!|0x!oct2!" 56 | if "%1"=="/xor" set /a bitwise3="0x!oct1!^^0x!oct2!" 57 | call :dec2hex !bitwise3! bitwise3 58 | set bitwise4=!bitwise3:~2!!bitwise4! 59 | if "!bitwise1!"=="" if "!bitwise2!"=="" ( 60 | call :zeros !bitwise4! bitwise4 61 | echo 0x!bitwise4! 62 | exit /b 63 | ) 64 | goto bin_loop 65 | 66 | rem Sign detection function to designate positive or negative 67 | :detect_sign 68 | set string=%~1 69 | if "!string:~,1!"=="-" (set %2=-) else (set %2=+) 70 | exit /b 71 | 72 | rem Sign stripping function 73 | :remove_sign 74 | set string=%~1 75 | if "!string:~,1!"=="-" (set %2=!string:~1!) else (set %2=!string!) 76 | exit /b 77 | 78 | rem Positive on positive and nagative on negative addition calculations 79 | :add_calc 80 | call :shift_right "!string1!" string1 oct1 81 | call :shift_right "!string2!" string2 oct2 82 | call :zeros !oct1! oct1 83 | call :zeros !oct2! oct2 84 | set /a oct3=100000000+!oct1!+!oct2!+!digit4! 85 | set string3=!oct3:~-8!!string3! 86 | if !oct3! geq 200000000 (set digit4=1) else (set digit4=0) 87 | if "!string1!"=="" if "!string2!"=="" if "!digit4!"=="0" ( 88 | call :zeros !string3! string3 89 | if "!sign1!"=="-" set string3=-!string3! 90 | exit /b 91 | ) 92 | goto add_calc 93 | 94 | rem Positive on negative subtraction calculations 95 | :subtract_calc 96 | call :eval !string1! !string2! string 97 | 98 | if "!string!"=="eq" ( 99 | set string3=0 100 | exit /b 101 | ) 102 | 103 | if "!string!"=="gtr" ( 104 | set sign=!sign1! 105 | ) else ( 106 | set sign=!sign2! 107 | set string=!string1! 108 | set string1=!string2! 109 | set string2=!string! 110 | ) 111 | 112 | :subtract_loop 113 | call :shift_right "!string1!" string1 oct1 114 | call :shift_right "!string2!" string2 oct2 115 | call :zeros !oct1! oct1 116 | call :zeros !oct2! oct2 117 | set /a oct3=100000000+!oct1!-!oct2!-!digit4! 118 | if !oct3! lss 100000000 (set /a oct3=!oct3!+100000000&set digit4=1) else (set digit4=0) 119 | set string3=!oct3:~-8!!string3! 120 | if "!string1!"=="" if "!string2!"=="" ( 121 | call :zeros !string3! string3 122 | if "!sign!"=="-" set string3=-!string3! 123 | exit /b 124 | ) 125 | goto subtract_loop 126 | 127 | rem Evaluate two numbers 128 | rem This function is a necessity to replace native evaluations for "if X gtr/lss/eq Y" for numbers larger than 32 bits 129 | rem For "or equal to" evaluations, just use "if X not gtr/lss/eq Y" and pick the one you are NOT looking for of the three choices 130 | :eval 131 | if "%1"=="%2" ( 132 | call :return eq %3 133 | exit /b 134 | ) 135 | 136 | if "!sign1!"=="" if "!sign2!"=="" ( 137 | call :detect_sign "%1" sign1 138 | call :detect_sign "%2" sign2 139 | 140 | if not "!sign1!"=="!sign2!" ( 141 | if "!sign1!"=="-" ( 142 | call :return lss %3 143 | ) else ( 144 | call :return gtr %3 145 | ) 146 | exit /b 147 | ) 148 | ) 149 | 150 | call :count_digits %1 count1 151 | call :count_digits %2 count2 152 | 153 | if not !count1!==!count2! ( 154 | if !count1! gtr !count2! ( 155 | call :return gtr %3 156 | ) else ( 157 | call :return lss %3) 158 | exit /b 159 | ) 160 | 161 | call :sort_digits %1 %2 162 | 163 | if "!string!"=="%1" ( 164 | call :return gtr %3 165 | ) else ( 166 | call :return lss %3 167 | ) 168 | exit /b 169 | 170 | rem Return function to handle interactive or scripted mode 171 | :return 172 | if "%2"=="" (echo %1) else (set %2=%1) 173 | exit /b 174 | 175 | rem Count how many characters in a string 176 | :count_digits 177 | set count= 178 | call :expand_string %1 string 179 | for %%0 in (!string!) do set /a count=!count!+1 180 | set %2=!count! 181 | exit /b 182 | 183 | rem Sort automatically evaluates strings character by character going from left to right 184 | rem Useful for evaluating numbers of the same character length quickly 185 | :sort_digits 186 | for /f %%0 in ('^(echo %1^&echo %2^) ^| sort') do set string=%%0 187 | exit /b 188 | 189 | rem Separate number characters into substrings to iterate character-to-character calculations in for loops 190 | :expand_string 191 | set string=%1 192 | set string=!string:0= 0 ! 193 | set string=!string:1= 1 ! 194 | set string=!string:2= 2 ! 195 | set string=!string:3= 3 ! 196 | set string=!string:4= 4 ! 197 | set string=!string:5= 5 ! 198 | set string=!string:6= 6 ! 199 | set string=!string:7= 7 ! 200 | set string=!string:8= 8 ! 201 | set string=!string:9= 9 ! 202 | set string=!string:A= A ! 203 | set string=!string:B= B ! 204 | set string=!string:C= C ! 205 | set string=!string:D= D ! 206 | set string=!string:E= E ! 207 | set string=!string:F= F ! 208 | set %2=!string! 209 | exit /b 210 | 211 | rem Condense an expanded number back to a single string 212 | :condense_string 213 | set string=%~1 214 | set string=!string: =! 215 | set %2=!string! 216 | exit /b 217 | 218 | rem Segments a string to be calculated in groups of 8 characters at a time 219 | :shift_right 220 | if "%~1"=="" ( 221 | set %3=0 222 | ) else ( 223 | set string=%~1 224 | if "!string:~,-8!"=="" ( 225 | set %~3=!string! 226 | set %~2= 227 | ) else ( 228 | set %3=!string:~-8! 229 | set %2=!string:~,-8! 230 | ) 231 | ) 232 | exit /b 233 | 234 | rem Strip leading zeros 235 | rem Example: 010 -> stripped -> 10 -> reversed -> 01 -> stripped -> 1 236 | :zeros 237 | if "%1"=="0" exit /b 238 | set string=%1 239 | :zeros_loop 240 | if "!string:~,1!"=="0" if not "!string!"=="0" ( 241 | set string=!string:~1! 242 | goto zeros_loop 243 | ) 244 | set %2=!string! 245 | exit /b 246 | 247 | rem Reverse an expanded number to calculate substring characters backwards 248 | :reverse_expanded 249 | set string= 250 | for %%0 in (%~1) do set string=%%0 !string! 251 | set %2=!string! 252 | exit /b 253 | 254 | rem Convert decimal to hexidecimal 255 | :dec2hex 256 | if !external!==1 set external=2 257 | call :dec2bin %1 hex 258 | call :bin2hex !hex! hex 259 | if !external!==2 (echo !hex!) else (set %2=!hex!) 260 | exit /b 261 | 262 | rem Convert binary to hexidecimal 263 | :bin2hex 264 | if "%1"=="" ( 265 | set %2=0x0 266 | exit /b 267 | ) 268 | set string=%1 269 | set hex= 270 | :bin2hex_loop 271 | if "!string:~,-4!"=="" ( 272 | set nibble=!string! 273 | set string= 274 | ) else ( 275 | set nibble=!string:~-4! 276 | set string=!string:~,-4! 277 | ) 278 | 279 | if "!nibble!"=="0" set nibble=0 280 | if "!nibble!"=="1" set nibble=1 281 | 282 | if "!nibble!"=="00" set nibble=0 283 | if "!nibble!"=="01" set nibble=1 284 | if "!nibble!"=="10" set nibble=2 285 | if "!nibble!"=="11" set nibble=3 286 | 287 | if "!nibble!"=="000" set nibble=0 288 | if "!nibble!"=="001" set nibble=1 289 | if "!nibble!"=="010" set nibble=2 290 | if "!nibble!"=="011" set nibble=3 291 | if "!nibble!"=="100" set nibble=4 292 | if "!nibble!"=="101" set nibble=5 293 | if "!nibble!"=="110" set nibble=6 294 | if "!nibble!"=="111" set nibble=7 295 | 296 | if "!nibble!"=="0000" set nibble=0 297 | if "!nibble!"=="0001" set nibble=1 298 | if "!nibble!"=="0010" set nibble=2 299 | if "!nibble!"=="0011" set nibble=3 300 | if "!nibble!"=="0100" set nibble=4 301 | if "!nibble!"=="0101" set nibble=5 302 | if "!nibble!"=="0110" set nibble=6 303 | if "!nibble!"=="0111" set nibble=7 304 | if "!nibble!"=="1000" set nibble=8 305 | if "!nibble!"=="1001" set nibble=9 306 | if "!nibble!"=="1010" set nibble=A 307 | if "!nibble!"=="1011" set nibble=B 308 | if "!nibble!"=="1100" set nibble=C 309 | if "!nibble!"=="1101" set nibble=D 310 | if "!nibble!"=="1110" set nibble=E 311 | if "!nibble!"=="1111" set nibble=F 312 | 313 | set hex=!nibble!!hex! 314 | if "!string!"=="" ( 315 | set %2=0x!hex! 316 | exit /b 317 | ) 318 | goto bin2hex_loop 319 | 320 | rem Convert hexidecimal to binary 321 | :hex2bin 322 | if "%1"=="" ( 323 | set %2=0 324 | exit /b 325 | ) 326 | 327 | set string=%1 328 | set string=!string:0x=! 329 | set string=!string:0=0000! 330 | set string=!string:1=0001! 331 | set string=!string:2=0010! 332 | set string=!string:3=0011! 333 | set string=!string:4=0100! 334 | set string=!string:5=0101! 335 | set string=!string:6=0110! 336 | set string=!string:7=0111! 337 | set string=!string:8=1000! 338 | set string=!string:9=1001! 339 | set string=!string:A=1010! 340 | set string=!string:B=1011! 341 | set string=!string:C=1100! 342 | set string=!string:D=1101! 343 | set string=!string:E=1110! 344 | set string=!string:F=1111! 345 | set %2=!string! 346 | exit /b 347 | 348 | rem If a 32-bit number is negative, this will break it out to be a true 32-bit number with no bit for sign control 349 | :32bit 350 | call :detect_sign "%1" sign 351 | call :remove_sign "%1" dec 352 | if "!sign!"=="-" (call :add 4294967296 %~1 dec) 353 | echo !dec! 354 | exit /b 355 | 356 | rem If a 32-bit positive number takes up 32 bits, this converts it to a 32-bit negative with sign control 357 | :32bitp 358 | set dec=%1 359 | call :eval %1 2147483648 string 360 | if not "!string!"=="lss" (call :add -4294967296 %~1 dec) 361 | echo !dec! 362 | exit /b 363 | 364 | rem Converts decimal to binary 365 | :dec2bin 366 | if "%1"=="0" ( 367 | if "!external!"=="1" (echo 0) else (set %2=0) 368 | exit /b 369 | ) 370 | 371 | call :detect_sign "%1" sign 372 | call :remove_sign "%1" dec 373 | if "!sign!"=="-" (call :add 4294967296 %~1 dec) 374 | 375 | rem Load default 32-bit number set 376 | 377 | set bits=2147483648 378 | set bit=2147483648 379 | 380 | rem Load 64-bit number set if the number is larger than 32 bits 381 | 382 | call :eval !dec! 2147483648 string 383 | if "!string!"=="gtr" ( 384 | set bits=9223372036854775808 4611686018427387904 2305843009213693952 1152921504606846976 576460752303423488 288230376151711744 144115188075855872 72057594037927936 36028797018963968 18014398509481984 9007199254740992 4503599627370496 2251799813685248 1125899906842624 562949953421312 281474976710656 140737488355328 70368744177664 35184372088832 17592186044416 8796093022208 4398046511104 2199023255552 1099511627776 549755813888 274877906944 137438953472 68719476736 34359738368 17179869184 8589934592 4294967296 !bits! 385 | set bit=9223372036854775808 386 | 387 | rem Load 128-bit number set if the number is larger than 64 bits 388 | 389 | call :eval !dec! 9223372036854775808 string 390 | if "!string!"=="gtr" ( 391 | set bits=170141183460469231731687303715884105728 85070591730234615865843651857942052864 42535295865117307932921825928971026432 21267647932558653966460912964485513216 10633823966279326983230456482242756608 5316911983139663491615228241121378304 2658455991569831745807614120560689152 1329227995784915872903807060280344576 664613997892457936451903530140172288 332306998946228968225951765070086144 166153499473114484112975882535043072 83076749736557242056487941267521536 41538374868278621028243970633760768 20769187434139310514121985316880384 10384593717069655257060992658440192 5192296858534827628530496329220096 2596148429267413814265248164610048 1298074214633706907132624082305024 649037107316853453566312041152512 324518553658426726783156020576256 162259276829213363391578010288128 81129638414606681695789005144064 40564819207303340847894502572032 20282409603651670423947251286016 10141204801825835211973625643008 5070602400912917605986812821504 2535301200456458802993406410752 1267650600228229401496703205376 633825300114114700748351602688 316912650057057350374175801344 158456325028528675187087900672 79228162514264337593543950336 39614081257132168796771975168 19807040628566084398385987584 9903520314283042199192993792 4951760157141521099596496896 2475880078570760549798248448 1237940039285380274899124224 618970019642690137449562112 309485009821345068724781056 154742504910672534362390528 77371252455336267181195264 38685626227668133590597632 19342813113834066795298816 9671406556917033397649408 4835703278458516698824704 2417851639229258349412352 1208925819614629174706176 604462909807314587353088 302231454903657293676544 151115727451828646838272 75557863725914323419136 37778931862957161709568 18889465931478580854784 9444732965739290427392 4722366482869645213696 2361183241434822606848 1180591620717411303424 590295810358705651712 295147905179352825856 147573952589676412928 73786976294838206464 36893488147419103232 18446744073709551616 !bits! 392 | set bit=170141183460469231731687303715884105728 393 | 394 | rem If the number is higher than 128 bits, check if there is a cache file with a higher number set 395 | rem If the cache file exists, load the number set 396 | 397 | call :eval !dec! 170141183460469231731687303715884105728 string 398 | if "!string!"=="gtr" ( 399 | if exist !bdat! ( 400 | set /p bits=!bdat! 424 | set cache= 425 | ) 426 | 427 | rem Begin evaluating input number against number set to calculate bits 428 | :dec2bin_return 429 | set bin= 430 | 431 | rem Loop to calculate numbers 32 bits and higher 432 | 433 | for %%0 in (!bits!) do ( 434 | call :eval !dec! %%0 string 435 | if "!string!"=="lss" (set bin=!bin!0) else (set bin=!bin!1&call :add !dec! -%%0 dec) 436 | ) 437 | 438 | rem Loop to calculate 31 bits and lower 439 | 440 | for /l %%0 in (30,-1,0) do ( 441 | set /a bit="!dec!&(1<<%%0)" 442 | if !bit!==0 (set bin=!bin!0) else (set bin=!bin!1) 443 | ) 444 | 445 | rem Clean up leading zeros and return result 446 | 447 | call :zeros !bin! bin 448 | if "!external!"=="1" (echo !bin!) else (set %2=!bin!) 449 | exit /b 450 | -------------------------------------------------------------------------------- /wget/COPYING: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /wget/README.md: -------------------------------------------------------------------------------- 1 | WGET 2 | 3 | NAME 4 | Wget − The non−interactive network downloader. 5 | 6 | SYNOPSIS 7 | wget [option]... [ URL ]... 8 | 9 | DESCRIPTION 10 | GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies. 11 | 12 | Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data. 13 | 14 | Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as "recursive downloading." While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing. 15 | 16 | Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off. 17 | 18 | OPTIONS 19 | Option Syntax 20 | Since Wget uses GNU getopt to process command-line arguments, every option has a long form along with the short one. Long options are more convenient to remember, but take time to type. You may freely mix different option styles, or specify options after the command-line arguments. Thus you may write: 21 | 22 | wget −r −−tries=10 http://fly.srk.fer.hr/ −o log 23 | The space between the option accepting an argument and the argument may be omitted. Instead of −o log you can write −olog. 24 | 25 | You may put several options that do not require arguments together, like: 26 | 27 | wget −drc 28 | This is completely equivalent to: 29 | 30 | wget −d −r −c 31 | Since the options can be specified after the arguments, you may terminate them with −−. So the following will try to download URL −x, reporting failure to log: 32 | 33 | wget −o log −− −x 34 | The options that accept comma-separated lists all respect the convention that specifying an empty list clears its value. This can be useful to clear the .wgetrc settings. For instance, if your .wgetrc sets "exclude_directories" to /cgi−bin, the following example will first reset it, and then set it to exclude /~nobody and /~somebody. You can also clear the lists in .wgetrc. 35 | 36 | wget −X " −X /~nobody,/~somebody 37 | Most options that do not accept arguments are boolean options, so named because their state can be captured with a yes-or-no ("boolean") variable. For example, −−follow−ftp tells Wget to follow FTP links from HTML files and, on the other hand, −−no−glob tells it not to perform file globbing on FTP URLs. A boolean option is either affirmative or negative (beginning with −−no). All such options share several properties. 38 | 39 | Unless stated otherwise, it is assumed that the default behavior is the opposite of what the option accomplishes. For example, the documented existence of −−follow−ftp assumes that the default is to not follow FTP links from HTML pages. 40 | 41 | Affirmative options can be negated by prepending the −−no− to the option name; negative options can be negated by omitting the −−no− prefix. This might seem superfluous−−−if the default for an affirmative option is to not do something, then why provide a way to explicitly turn it off? But the startup file may in fact change the default. For instance, using "follow_ftp = on" in .wgetrc makes Wget follow FTP links by default, and using −−no−follow−ftp is the only way to restore the factory default from the command line. 42 | 43 | Basic Startup Options 44 | 45 | −V 46 | 47 | −−version 48 | 49 | Display the version of Wget. 50 | 51 | −h 52 | 53 | −−help 54 | 55 | Print a help message describing all of Wget’s command-line options. 56 | 57 | −b 58 | 59 | −−background 60 | 61 | Go to background immediately after startup. If no output file is specified via the −o, output is redirected to wget-log. 62 | 63 | −e command 64 | −−execute command 65 | 66 | Execute command as if it were a part of .wgetrc. A command thus invoked will be executed after the commands in .wgetrc, thus taking precedence over them. If you need to specify more than one wgetrc command, use multiple instances of −e. 67 | 68 | Logging and Input File Options 69 | −o logfile 70 | −−output−file=logfile 71 | 72 | Log all messages to logfile. The messages are normally reported to standard error. 73 | 74 | −a logfile 75 | −−append−output=logfile 76 | 77 | Append to logfile. This is the same as −o, only it appends to logfile instead of overwriting the old log file. If logfile does not exist, a new file is created. 78 | 79 | −d 80 | 81 | −−debug 82 | 83 | Turn on debug output, meaning various information important to the developers of Wget if it does not work properly. Your system administrator may have chosen to compile Wget without debug support, in which case −d will not work. Please note that compiling with debug support is always safe−−−Wget compiled with the debug support will not print any debug info unless requested with −d. 84 | 85 | −q 86 | 87 | −−quiet 88 | 89 | Turn off Wget’s output. 90 | 91 | −v 92 | 93 | −−verbose 94 | 95 | Turn on verbose output, with all the available data. The default output is verbose. 96 | 97 | −nv 98 | 99 | −−no−verbose 100 | 101 | Turn off verbose without being completely quiet (use −q for that), which means that error messages and basic information still get printed. 102 | 103 | −−report−speed=type 104 | 105 | Output bandwidth as type. The only accepted value is bits. 106 | 107 | −i file 108 | −−input−file=file 109 | 110 | Read URLs from a local or external file. If − is specified as file, URLs are read from the standard input. (Use ./− to read from a file literally named −.) 111 | 112 | If this function is used, no URLs need be present on the command line. If there are URLs both on the command line and in an input file, those on the command lines will be the first ones to be retrieved. If −−force−html is not specified, then file should consist of a series of URLs, one per line. 113 | 114 | However, if you specify −−force−html, the document will be regarded as html. In that case you may have problems with relative links, which you can solve either by adding "" to the documents or by specifying −−base=url on the command line. 115 | 116 | If the file is an external one, the document will be automatically treated as html if the Content-Type matches text/html. Furthermore, the file’s location will be implicitly used as base href if none was specified. 117 | 118 | −−input−metalink=file 119 | 120 | Downloads files covered in local Metalink file. Metalink version 3 and 4 are supported. 121 | 122 | −−keep−badhash 123 | 124 | Keeps downloaded Metalink’s files with a bad hash. It appends .badhash to the name of Metalink’s files which have a checksum mismatch, except without overwriting existing files. 125 | 126 | −−metalink−over−http 127 | 128 | Issues HTTP HEAD request instead of GET and extracts Metalink metadata from response headers. Then it switches to Metalink download. If no valid Metalink metadata is found, it falls back to ordinary HTTP download. Enables Content-Type: application/metalink4+xml files download/processing. 129 | 130 | −−metalink−index=number 131 | 132 | Set the Metalink application/metalink4+xml metaurl ordinal NUMBER. From 1 to the total number of "application/metalink4+xml" available. Specify 0 or inf to choose the first good one. Metaurls, such as those from a −−metalink−over−http, may have been sorted by priority key’s value; keep this in mind to choose the right NUMBER. 133 | 134 | −−preferred−location 135 | 136 | Set preferred location for Metalink resources. This has effect if multiple resources with same priority are available. 137 | 138 | −F 139 | 140 | −−force−html 141 | 142 | When input is read from a file, force it to be treated as an HTML file. This enables you to retrieve relative links from existing HTML files on your local disk, by adding "" to HTML, or using the −−base command-line option. 143 | 144 | −B URL 145 | −−base= URL 146 | 147 | Resolves relative links using URL as the point of reference, when reading links from an HTML file specified via the −i/−−input−file option (together with −−force−html, or when the input file was fetched remotely from a server describing it as HTML ). This is equivalent to the presence of a "BASE" tag in the HTML input file, with URL as the value for the "href" attribute. 148 | 149 | For instance, if you specify http://foo/bar/a.html for URL , and Wget reads ../baz/b.html from the input file, it would be resolved to http://foo/baz/b.html. 150 | 151 | −−config= FILE 152 | 153 | Specify the location of a startup file you wish to use instead of the default one(s). Use −−no−config to disable reading of config files. If both −−config and −−no−config are given, −−no−config is ignored. 154 | 155 | −−rejected−log=logfile 156 | 157 | Logs all URL rejections to logfile as comma separated values. The values include the reason of rejection, the URL and the parent URL it was found in. 158 | 159 | Download Options 160 | −−bind−address= ADDRESS 161 | 162 | When making client TCP/IP connections, bind to ADDRESS on the local machine. ADDRESS may be specified as a hostname or IP address. This option can be useful if your machine is bound to multiple IPs. 163 | 164 | −−bind−dns−address= ADDRESS 165 | 166 | [libcares only] This address overrides the route for DNS requests. If you ever need to circumvent the standard settings from /etc/resolv.conf, this option together with −−dns−servers is your friend. ADDRESS must be specified either as IPv4 or IPv6 address. Wget needs to be built with libcares for this option to be available. 167 | 168 | −−dns−servers= ADDRESSES 169 | 170 | [libcares only] The given address(es) override the standard nameserver addresses, e.g. as configured in /etc/resolv.conf. ADDRESSES may be specified either as IPv4 or IPv6 addresses, comma-separated. Wget needs to be built with libcares for this option to be available. 171 | 172 | −t number 173 | −−tries=number 174 | 175 | Set number of tries to number. Specify 0 or inf for infinite retrying. The default is to retry 20 times, with the exception of fatal errors like "connection refused" or "not found" (404), which are not retried. 176 | 177 | −O file 178 | −−output−document=file 179 | 180 | The documents will not be written to the appropriate files, but all will be concatenated together and written to file. If − is used as file, documents will be printed to standard output, disabling link conversion. (Use ./− to print to a file literally named −.) 181 | 182 | Use of −O is not intended to mean simply "use the name file instead of the one in the URL ;" rather, it is analogous to shell redirection: wget −O file http://foo is intended to work like wget −O − http://foo > file; file will be truncated immediately, and all downloaded content will be written there. 183 | 184 | For this reason, −N (for timestamp-checking) is not supported in combination with −O: since file is always newly created, it will always have a very new timestamp. A warning will be issued if this combination is used. 185 | 186 | Similarly, using −r or −p with −O may not work as you expect: Wget won’t just download the first file to file and then download the rest to their normal names: all downloaded content will be placed in file. This was disabled in version 1.11, but has been reinstated (with a warning) in 1.11.2, as there are some cases where this behavior can actually have some use. 187 | 188 | A combination with −nc is only accepted if the given output file does not exist. 189 | 190 | Note that a combination with −k is only permitted when downloading a single document, as in that case it will just convert all relative URIs to external ones; −k makes no sense for multiple URIs when they’re all being downloaded to a single file; −k can be used only when the output is a regular file. 191 | 192 | −nc 193 | 194 | −−no−clobber 195 | 196 | If a file is downloaded more than once in the same directory, Wget’s behavior depends on a few options, including −nc. In certain cases, the local file will be clobbered, or overwritten, upon repeated download. In other cases it will be preserved. 197 | 198 | When running Wget without −N, −nc, −r, or −p, downloading the same file in the same directory will result in the original copy of file being preserved and the second copy being named file.1. If that file is downloaded yet again, the third copy will be named file.2, and so on. (This is also the behavior with −nd, even if −r or −p are in effect.) When −nc is specified, this behavior is suppressed, and Wget will refuse to download newer copies of file. Therefore, ""no−clobber"" is actually a misnomer in this mode−−−it’s not clobbering that’s prevented (as the numeric suffixes were already preventing clobbering), but rather the multiple version saving that’s prevented. 199 | 200 | When running Wget with −r or −p, but without −N, −nd, or −nc, re-downloading a file will result in the new copy simply overwriting the old. Adding −nc will prevent this behavior, instead causing the original version to be preserved and any newer copies on the server to be ignored. 201 | 202 | When running Wget with −N, with or without −r or −p, the decision as to whether or not to download a newer copy of a file depends on the local and remote timestamp and size of the file. −nc may not be specified at the same time as −N. 203 | 204 | A combination with −O/−−output−document is only accepted if the given output file does not exist. 205 | 206 | Note that when −nc is specified, files with the suffixes .html or .htm will be loaded from the local disk and parsed as if they had been retrieved from the Web. 207 | 208 | −−backups=backups 209 | 210 | Before (over)writing a file, back up an existing file by adding a .1 suffix (_1 on VMS ) to the file name. Such backup files are rotated to .2, .3, and so on, up to backups (and lost beyond that). 211 | 212 | −−no−netrc 213 | 214 | Do not try to obtain credentials from .netrc file. By default .netrc file is searched for credentials in case none have been passed on command line and authentication is required. 215 | 216 | −c 217 | 218 | −−continue 219 | 220 | Continue getting a partially-downloaded file. This is useful when you want to finish up a download started by a previous instance of Wget, or by another program. For instance: 221 | 222 | wget −c ftp://sunsite.doc.ic.ac.uk/ls−lR.Z 223 | If there is a file named ls−lR.Z in the current directory, Wget will assume that it is the first portion of the remote file, and will ask the server to continue the retrieval from an offset equal to the length of the local file. 224 | 225 | Note that you don’t need to specify this option if you just want the current invocation of Wget to retry downloading a file should the connection be lost midway through. This is the default behavior. −c only affects resumption of downloads started prior to this invocation of Wget, and whose local files are still sitting around. 226 | 227 | Without −c, the previous example would just download the remote file to ls−lR.Z.1, leaving the truncated ls−lR.Z file alone. 228 | 229 | If you use −c on a non-empty file, and the server does not support continued downloading, Wget will restart the download from scratch and overwrite the existing file entirely. 230 | 231 | Beginning with Wget 1.7, if you use −c on a file which is of equal size as the one on the server, Wget will refuse to download the file and print an explanatory message. The same happens when the file is smaller on the server than locally (presumably because it was changed on the server since your last download attempt)−−−because "continuing" is not meaningful, no download occurs. 232 | 233 | On the other side of the coin, while using −c, any file that’s bigger on the server than locally will be considered an incomplete download and only "(length(remote) − length(local))" bytes will be downloaded and tacked onto the end of the local file. This behavior can be desirable in certain cases−−−for instance, you can use wget −c to download just the new portion that’s been appended to a data collection or log file. 234 | 235 | However, if the file is bigger on the server because it’s been changed, as opposed to just appended to, you’ll end up with a garbled file. Wget has no way of verifying that the local file is really a valid prefix of the remote file. You need to be especially careful of this when using −c in conjunction with −r, since every file will be considered as an "incomplete download" candidate. 236 | 237 | Another instance where you’ll get a garbled file if you try to use −c is if you have a lame HTTP proxy that inserts a "transfer interrupted" string into the local file. In the future a "rollback" option may be added to deal with this case. 238 | 239 | Note that −c only works with FTP servers and with HTTP servers that support the "Range" header. 240 | 241 | −−start−pos= OFFSET 242 | 243 | Start downloading at zero-based position OFFSET . Offset may be expressed in bytes, kilobytes with the ‘k’ suffix, or megabytes with the ‘m’ suffix, etc. 244 | 245 | −−start−pos has higher precedence over −−continue. When −−start−pos and −−continue are both specified, wget will emit a warning then proceed as if −−continue was absent. 246 | 247 | Server support for continued download is required, otherwise −−start−pos cannot help. See −c for details. 248 | 249 | −−progress=type 250 | 251 | Select the type of the progress indicator you wish to use. Legal indicators are "dot" and "bar". 252 | 253 | The "bar" indicator is used by default. It draws an ASCII progress bar graphics (a.k.a "thermometer" display) indicating the status of retrieval. If the output is not a TTY, the "dot" bar will be used by default. 254 | 255 | Use −−progress=dot to switch to the "dot" display. It traces the retrieval by printing dots on the screen, each dot representing a fixed amount of downloaded data. 256 | 257 | The progress type can also take one or more parameters. The parameters vary based on the type selected. Parameters to type are passed by appending them to the type sperated by a colon (:) like this: −−progress=type:parameter1:parameter2. 258 | 259 | When using the dotted retrieval, you may set the style by specifying the type as dot:style. Different styles assign different meaning to one dot. With the "default" style each dot represents 1K, there are ten dots in a cluster and 50 dots in a line. The "binary" style has a more "computer"−like orientation−−−8K dots, 16−dots clusters and 48 dots per line (which makes for 384K lines). The "mega" style is suitable for downloading large files−−−each dot represents 64K retrieved, there are eight dots in a cluster, and 48 dots on each line (so each line contains 3M). If "mega" is not enough then you can use the "giga" style−−−each dot represents 1M retrieved, there are eight dots in a cluster, and 32 dots on each line (so each line contains 32M). 260 | 261 | With −−progress=bar, there are currently two possible parameters, force and noscroll. 262 | 263 | When the output is not a TTY, the progress bar always falls back to "dot", even if −−progress=bar was passed to Wget during invocation. This behaviour can be overridden and the "bar" output forced by using the "force" parameter as −−progress=bar:force. 264 | 265 | By default, the bar style progress bar scroll the name of the file from left to right for the file being downloaded if the filename exceeds the maximum length allotted for its display. In certain cases, such as with −−progress=bar:force, one may not want the scrolling filename in the progress bar. By passing the "noscroll" parameter, Wget can be forced to display as much of the filename as possible without scrolling through it. 266 | 267 | Note that you can set the default style using the "progress" command in .wgetrc. That setting may be overridden from the command line. For example, to force the bar output without scrolling, use −−progress=bar:force:noscroll. 268 | 269 | −−show−progress 270 | 271 | Force wget to display the progress bar in any verbosity. 272 | 273 | By default, wget only displays the progress bar in verbose mode. One may however, want wget to display the progress bar on screen in conjunction with any other verbosity modes like −−no−verbose or −−quiet. This is often a desired a property when invoking wget to download several small/large files. In such a case, wget could simply be invoked with this parameter to get a much cleaner output on the screen. 274 | 275 | This option will also force the progress bar to be printed to stderr when used alongside the −−logfile option. 276 | 277 | −N 278 | 279 | −−timestamping 280 | 281 | Turn on time-stamping. 282 | 283 | −−no−if−modified−since 284 | 285 | Do not send If-Modified-Since header in −N mode. Send preliminary HEAD request instead. This has only effect in −N mode. 286 | 287 | −−no−use−server−timestamps 288 | 289 | Don’t set the local file’s timestamp by the one on the server. 290 | 291 | By default, when a file is downloaded, its timestamps are set to match those from the remote file. This allows the use of −−timestamping on subsequent invocations of wget. However, it is sometimes useful to base the local file’s timestamp on when it was actually downloaded; for that purpose, the −−no−use−server−timestamps option has been provided. 292 | 293 | −S 294 | 295 | −−server−response 296 | 297 | Print the headers sent by HTTP servers and responses sent by FTP servers. 298 | 299 | −−spider 300 | 301 | When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there. For example, you can use Wget to check your bookmarks: 302 | 303 | wget −−spider −−force−html −i bookmarks.html 304 | This feature needs much more work for Wget to get close to the functionality of real web spiders. 305 | 306 | −T seconds 307 | −−timeout=seconds 308 | 309 | Set the network timeout to seconds seconds. This is equivalent to specifying −−dns−timeout, −−connect−timeout, and −−read−timeout, all at the same time. 310 | 311 | When interacting with the network, Wget can check for timeout and abort the operation if it takes too long. This prevents anomalies like hanging reads and infinite connects. The only timeout enabled by default is a 900−second read timeout. Setting a timeout to 0 disables it altogether. Unless you know what you are doing, it is best not to change the default timeout settings. 312 | 313 | All timeout-related options accept decimal values, as well as subsecond values. For example, 0.1 seconds is a legal (though unwise) choice of timeout. Subsecond timeouts are useful for checking server response times or for testing network latency. 314 | 315 | −−dns−timeout=seconds 316 | 317 | Set the DNS lookup timeout to seconds seconds. DNS lookups that don’t complete within the specified time will fail. By default, there is no timeout on DNS lookups, other than that implemented by system libraries. 318 | 319 | −−connect−timeout=seconds 320 | 321 | Set the connect timeout to seconds seconds. TCP connections that take longer to establish will be aborted. By default, there is no connect timeout, other than that implemented by system libraries. 322 | 323 | −−read−timeout=seconds 324 | 325 | Set the read (and write) timeout to seconds seconds. The "time" of this timeout refers to idle time: if, at any point in the download, no data is received for more than the specified number of seconds, reading fails and the download is restarted. This option does not directly affect the duration of the entire download. 326 | 327 | Of course, the remote server may choose to terminate the connection sooner than this option requires. The default read timeout is 900 seconds. 328 | 329 | −−limit−rate=amount 330 | 331 | Limit the download speed to amount bytes per second. Amount may be expressed in bytes, kilobytes with the k suffix, or megabytes with the m suffix. For example, −−limit−rate=20k will limit the retrieval rate to 20KB/s. This is useful when, for whatever reason, you don’t want Wget to consume the entire available bandwidth. 332 | 333 | This option allows the use of decimal numbers, usually in conjunction with power suffixes; for example, −−limit−rate=2.5k is a legal value. 334 | 335 | Note that Wget implements the limiting by sleeping the appropriate amount of time after a network read that took less time than specified by the rate. Eventually this strategy causes the TCP transfer to slow down to approximately the specified rate. However, it may take some time for this balance to be achieved, so don’t be surprised if limiting the rate doesn’t work well with very small files. 336 | 337 | −w seconds 338 | −−wait=seconds 339 | 340 | Wait the specified number of seconds between the retrievals. Use of this option is recommended, as it lightens the server load by making the requests less frequent. Instead of in seconds, the time can be specified in minutes using the "m" suffix, in hours using "h" suffix, or in days using "d" suffix. 341 | 342 | Specifying a large value for this option is useful if the network or the destination host is down, so that Wget can wait long enough to reasonably expect the network error to be fixed before the retry. The waiting interval specified by this function is influenced by "−−random−wait", which see. 343 | 344 | −−waitretry=seconds 345 | 346 | If you don’t want Wget to wait between every retrieval, but only between retries of failed downloads, you can use this option. Wget will use linear backoff, waiting 1 second after the first failure on a given file, then waiting 2 seconds after the second failure on that file, up to the maximum number of seconds you specify. 347 | 348 | By default, Wget will assume a value of 10 seconds. 349 | 350 | −−random−wait 351 | 352 | Some web sites may perform log analysis to identify retrieval programs such as Wget by looking for statistically significant similarities in the time between requests. This option causes the time between requests to vary between 0.5 and 1.5 * wait seconds, where wait was specified using the −−wait option, in order to mask Wget’s presence from such analysis. 353 | 354 | A 2001 article in a publication devoted to development on a popular consumer platform provided code to perform this analysis on the fly. Its author suggested blocking at the class C address level to ensure automated retrieval programs were blocked despite changing DHCP-supplied addresses. 355 | 356 | The −−random−wait option was inspired by this ill-advised recommendation to block many unrelated users from a web site due to the actions of one. 357 | 358 | −−no−proxy 359 | 360 | Don’t use proxies, even if the appropriate *_proxy environment variable is defined. 361 | 362 | −Q quota 363 | −−quota=quota 364 | 365 | Specify download quota for automatic retrievals. The value can be specified in bytes (default), kilobytes (with k suffix), or megabytes (with m suffix). 366 | 367 | Note that quota will never affect downloading a single file. So if you specify wget −Q10k https://example.com/ls−lR.gz, all of the ls−lR.gz will be downloaded. The same goes even when several URLs are specified on the command-line. However, quota is respected when retrieving either recursively, or from an input file. Thus you may safely type wget −Q2m −i sites−−−download will be aborted when the quota is exceeded. 368 | 369 | Setting quota to 0 or to inf unlimits the download quota. 370 | 371 | −−no−dns−cache 372 | 373 | Turn off caching of DNS lookups. Normally, Wget remembers the IP addresses it looked up from DNS so it doesn’t have to repeatedly contact the DNS server for the same (typically small) set of hosts it retrieves from. This cache exists in memory only; a new Wget run will contact DNS again. 374 | 375 | However, it has been reported that in some situations it is not desirable to cache host names, even for the duration of a short-running application like Wget. With this option Wget issues a new DNS lookup (more precisely, a new call to "gethostbyname" or "getaddrinfo") each time it makes a new connection. Please note that this option will not affect caching that might be performed by the resolving library or by an external caching layer, such as NSCD. 376 | 377 | If you don’t understand exactly what this option does, you probably won’t need it. 378 | 379 | −−restrict−file−names=modes 380 | 381 | Change which characters found in remote URLs must be escaped during generation of local filenames. Characters that are restricted by this option are escaped, i.e. replaced with %HH, where HH is the hexadecimal number that corresponds to the restricted character. This option may also be used to force all alphabetical cases to be either lower− or uppercase. 382 | 383 | By default, Wget escapes the characters that are not valid or safe as part of file names on your operating system, as well as control characters that are typically unprintable. This option is useful for changing these defaults, perhaps because you are downloading to a non-native partition, or because you want to disable escaping of the control characters, or you want to further restrict characters to only those in the ASCII range of values. 384 | 385 | The modes are a comma-separated set of text values. The acceptable values are unix, windows, nocontrol, ascii, lowercase, and uppercase. The values unix and windows are mutually exclusive (one will override the other), as are lowercase and uppercase. Those last are special cases, as they do not change the set of characters that would be escaped, but rather force local file paths to be converted either to lower− or uppercase. 386 | 387 | When "unix" is specified, Wget escapes the character / and the control characters in the ranges 0−−31 and 128−−159. This is the default on Unix-like operating systems. 388 | 389 | When "windows" is given, Wget escapes the characters \, |, /, :, ?, ", *, <, >, and the control characters in the ranges 0−−31 and 128−−159. In addition to this, Wget in Windows mode uses + instead of : to separate host and port in local file names, and uses @ instead of ? to separate the query portion of the file name from the rest. Therefore, a URL that would be saved as www.xemacs.org:4300/search.pl?input=blah in Unix mode would be saved as www.xemacs.org+4300/search.pl@input=blah in Windows mode. This mode is the default on Windows. 390 | 391 | If you specify nocontrol, then the escaping of the control characters is also switched off. This option may make sense when you are downloading URLs whose names contain UTF−8 characters, on a system which can save and display filenames in UTF−8 (some possible byte values used in UTF−8 byte sequences fall in the range of values designated by Wget as "controls"). 392 | 393 | The ascii mode is used to specify that any bytes whose values are outside the range of ASCII characters (that is, greater than 127) shall be escaped. This can be useful when saving filenames whose encoding does not match the one used locally. 394 | 395 | −4 396 | 397 | −−inet4−only 398 | 399 | −6 400 | 401 | −−inet6−only 402 | 403 | Force connecting to IPv4 or IPv6 addresses. With −−inet4−only or −4, Wget will only connect to IPv4 hosts, ignoring AAAA records in DNS, and refusing to connect to IPv6 addresses specified in URLs. Conversely, with −−inet6−only or −6, Wget will only connect to IPv6 hosts and ignore A records and IPv4 addresses. 404 | 405 | Neither options should be needed normally. By default, an IPv6−aware Wget will use the address family specified by the host’s DNS record. If the DNS responds with both IPv4 and IPv6 addresses, Wget will try them in sequence until it finds one it can connect to. (Also see "−−prefer−family" option described below.) 406 | 407 | These options can be used to deliberately force the use of IPv4 or IPv6 address families on dual family systems, usually to aid debugging or to deal with broken network configuration. Only one of −−inet6−only and −−inet4−only may be specified at the same time. Neither option is available in Wget compiled without IPv6 support. 408 | 409 | −−prefer−family=none/IPv4/IPv6 410 | 411 | When given a choice of several addresses, connect to the addresses with specified address family first. The address order returned by DNS is used without change by default. 412 | 413 | This avoids spurious errors and connect attempts when accessing hosts that resolve to both IPv6 and IPv4 addresses from IPv4 networks. For example, www.kame.net resolves to 2001:200:0:8002:203:47ff:fea5:3085 and to 203.178.141.194. When the preferred family is "IPv4", the IPv4 address is used first; when the preferred family is "IPv6", the IPv6 address is used first; if the specified value is "none", the address order returned by DNS is used without change. 414 | 415 | Unlike −4 and −6, this option doesn’t inhibit access to any address family, it only changes the order in which the addresses are accessed. Also note that the reordering performed by this option is stable−−−it doesn’t affect order of addresses of the same family. That is, the relative order of all IPv4 addresses and of all IPv6 addresses remains intact in all cases. 416 | 417 | −−retry−connrefused 418 | 419 | Consider "connection refused" a transient error and try again. Normally Wget gives up on a URL when it is unable to connect to the site because failure to connect is taken as a sign that the server is not running at all and that retries would not help. This option is for mirroring unreliable sites whose servers tend to disappear for short periods of time. 420 | 421 | −−user=user 422 | −−password=password 423 | 424 | Specify the username user and password password for both FTP and HTTP file retrieval. These parameters can be overridden using the −−ftp−user and −−ftp−password options for FTP connections and the −−http−user and −−http−password options for HTTP connections. 425 | 426 | −−ask−password 427 | 428 | Prompt for a password for each connection established. Cannot be specified when −−password is being used, because they are mutually exclusive. 429 | 430 | −−use−askpass=command 431 | 432 | Prompt for a user and password using the specified command. If no command is specified then the command in the environment variable WGET_ASKPASS is used. If WGET_ASKPASS is not set then the command in the environment variable SSH_ASKPASS is used. 433 | 434 | You can set the default command for use-askpass in the .wgetrc. That setting may be overridden from the command line. 435 | 436 | −−no−iri 437 | 438 | Turn off internationalized URI ( IRI ) support. Use −−iri to turn it on. IRI support is activated by default. 439 | 440 | You can set the default state of IRI support using the "iri" command in .wgetrc. That setting may be overridden from the command line. 441 | 442 | −−local−encoding=encoding 443 | 444 | Force Wget to use encoding as the default system encoding. That affects how Wget converts URLs specified as arguments from locale to UTF−8 for IRI support. 445 | 446 | Wget use the function "nl_langinfo()" and then the "CHARSET" environment variable to get the locale. If it fails, ASCII is used. 447 | 448 | You can set the default local encoding using the "local_encoding" command in .wgetrc. That setting may be overridden from the command line. 449 | 450 | −−remote−encoding=encoding 451 | 452 | Force Wget to use encoding as the default remote server encoding. That affects how Wget converts URIs found in files from remote encoding to UTF−8 during a recursive fetch. This options is only useful for IRI support, for the interpretation of non-ASCII characters. 453 | 454 | For HTTP, remote encoding can be found in HTTP "Content−Type" header and in HTML "Content−Type http−equiv" meta tag. 455 | 456 | You can set the default encoding using the "remoteencoding" command in .wgetrc. That setting may be overridden from the command line. 457 | 458 | −−unlink 459 | 460 | Force Wget to unlink file instead of clobbering existing file. This option is useful for downloading to the directory with hardlinks. 461 | 462 | Directory Options 463 | 464 | −nd 465 | 466 | −−no−directories 467 | 468 | Do not create a hierarchy of directories when retrieving recursively. With this option turned on, all files will get saved to the current directory, without clobbering (if a name shows up more than once, the filenames will get extensions .n). 469 | 470 | −x 471 | 472 | −−force−directories 473 | 474 | The opposite of −nd−−−create a hierarchy of directories, even if one would not have been created otherwise. E.g. wget −x http://fly.srk.fer.hr/robots.txt will save the downloaded file to fly.srk.fer.hr/robots.txt. 475 | 476 | −nH 477 | 478 | −−no−host−directories 479 | 480 | Disable generation of host-prefixed directories. By default, invoking Wget with −r http://fly.srk.fer.hr/ will create a structure of directories beginning with fly.srk.fer.hr/. This option disables such behavior. 481 | 482 | −−protocol−directories 483 | 484 | Use the protocol name as a directory component of local file names. For example, with this option, wget −r http://host will save to http/host/... rather than just to host/.... 485 | 486 | −−cut−dirs=number 487 | 488 | Ignore number directory components. This is useful for getting a fine-grained control over the directory where recursive retrieval will be saved. 489 | 490 | Take, for example, the directory at ftp://ftp.xemacs.org/pub/xemacs/. If you retrieve it with −r, it will be saved locally under ftp.xemacs.org/pub/xemacs/. While the −nH option can remove the ftp.xemacs.org/ part, you are still stuck with pub/xemacs. This is where −−cut−dirs comes in handy; it makes Wget not "see" number remote directory components. Here are several examples of how −−cut−dirs option works. 491 | 492 | No options −> ftp.xemacs.org/pub/xemacs/ 493 | −nH −> pub/xemacs/ 494 | −nH −−cut−dirs=1 −> xemacs/ 495 | −nH −−cut−dirs=2 −> . 496 | −−cut−dirs=1 −> ftp.xemacs.org/xemacs/ 497 | ... 498 | If you just want to get rid of the directory structure, this option is similar to a combination of −nd and −P. However, unlike −nd, −−cut−dirs does not lose with subdirectories−−−for instance, with −nH −−cut−dirs=1, a beta/ subdirectory will be placed to xemacs/beta, as one would expect. 499 | 500 | −P prefix 501 | −−directory−prefix=prefix 502 | 503 | Set directory prefix to prefix. The directory prefix is the directory where all other files and subdirectories will be saved to, i.e. the top of the retrieval tree. The default is . (the current directory). 504 | 505 | HTTP Options 506 | −−default−page=name 507 | 508 | Use name as the default file name when it isn’t known (i.e., for URLs that end in a slash), instead of index.html. 509 | 510 | −E 511 | 512 | −−adjust−extension 513 | 514 | If a file of type application/xhtml+xml or text/html is downloaded and the URL does not end with the regexp \.[Hh][Tt][Mm][Ll]?, this option will cause the suffix .html to be appended to the local filename. This is useful, for instance, when you’re mirroring a remote site that uses .asp pages, but you want the mirrored pages to be viewable on your stock Apache server. Another good use for this is when you’re downloading CGI-generated materials. A URL like http://site.com/article.cgi?25 will be saved as article.cgi?25.html. 515 | 516 | Note that filenames changed in this way will be re-downloaded every time you re-mirror a site, because Wget can’t tell that the local X.html file corresponds to remote URL X (since it doesn’t yet know that the URL produces output of type text/html or application/xhtml+xml. 517 | 518 | As of version 1.12, Wget will also ensure that any downloaded files of type text/css end in the suffix .css, and the option was renamed from −−html−extension, to better reflect its new behavior. The old option name is still acceptable, but should now be considered deprecated. 519 | 520 | As of version 1.19.2, Wget will also ensure that any downloaded files with a "Content−Encoding" of br, compress, deflate or gzip end in the suffix .br, .Z, .zlib and .gz respectively. 521 | 522 | At some point in the future, this option may well be expanded to include suffixes for other types of content, including content types that are not parsed by Wget. 523 | 524 | −−http−user=user 525 | −−http−password=password 526 | 527 | Specify the username user and password password on an HTTP server. According to the type of the challenge, Wget will encode them using either the "basic" (insecure), the "digest", or the Windows "NTLM" authentication scheme. 528 | 529 | Another way to specify username and password is in the URL itself. Either method reveals your password to anyone who bothers to run "ps". To prevent the passwords from being seen, use the −−use−askpass or store them in .wgetrc or .netrc, and make sure to protect those files from other users with "chmod". If the passwords are really important, do not leave them lying in those files either−−−edit the files and delete them after Wget has started the download. 530 | 531 | −−no−http−keep−alive 532 | 533 | Turn off the "keep-alive" feature for HTTP downloads. Normally, Wget asks the server to keep the connection open so that, when you download more than one document from the same server, they get transferred over the same TCP connection. This saves time and at the same time reduces the load on the server. 534 | 535 | This option is useful when, for some reason, persistent (keep-alive) connections don’t work for you, for example due to a server bug or due to the inability of server-side scripts to cope with the connections. 536 | 537 | −−no−cache 538 | 539 | Disable server-side cache. In this case, Wget will send the remote server an appropriate directive (Pragma: no-cache) to get the file from the remote service, rather than returning the cached version. This is especially useful for retrieving and flushing out-of-date documents on proxy servers. 540 | 541 | Caching is allowed by default. 542 | 543 | −−no−cookies 544 | 545 | Disable the use of cookies. Cookies are a mechanism for maintaining server-side state. The server sends the client a cookie using the "Set−Cookie" header, and the client responds with the same cookie upon further requests. Since cookies allow the server owners to keep track of visitors and for sites to exchange this information, some consider them a breach of privacy. The default is to use cookies; however, storing cookies is not on by default. 546 | 547 | −−load−cookies file 548 | 549 | Load cookies from file before the first HTTP retrieval. file is a textual file in the format originally used by Netscape’s cookies.txt file. 550 | 551 | You will typically use this option when mirroring sites that require that you be logged in to access some or all of their content. The login process typically works by the web server issuing an HTTP cookie upon receiving and verifying your credentials. The cookie is then resent by the browser when accessing that part of the site, and so proves your identity. 552 | 553 | Mirroring such a site requires Wget to send the same cookies your browser sends when communicating with the site. This is achieved by −−load−cookies−−−simply point Wget to the location of the cookies.txt file, and it will send the same cookies your browser would send in the same situation. Different browsers keep textual cookie files in different locations: 554 | "Netscape 4.x." 555 | 556 | The cookies are in ~/.netscape/cookies.txt. 557 | 558 | "Mozilla and Netscape 6.x." 559 | 560 | Mozilla’s cookie file is also named cookies.txt, located somewhere under ~/.mozilla, in the directory of your profile. The full path usually ends up looking somewhat like ~/.mozilla/default/some-weird-string/cookies.txt. 561 | 562 | "Internet Explorer." 563 | 564 | You can produce a cookie file Wget can use by using the File menu, Import and Export, Export Cookies. This has been tested with Internet Explorer 5; it is not guaranteed to work with earlier versions. 565 | 566 | "Other browsers." 567 | 568 | If you are using a different browser to create your cookies, −−load−cookies will only work if you can locate or produce a cookie file in the Netscape format that Wget expects. 569 | 570 | If you cannot use −−load−cookies, there might still be an alternative. If your browser supports a "cookie manager", you can use it to view the cookies used when accessing the site you’re mirroring. Write down the name and value of the cookie, and manually instruct Wget to send those cookies, bypassing the "official" cookie support: 571 | 572 | wget −−no−cookies −−header "Cookie: =" 573 | −−save−cookies file 574 | 575 | Save cookies to file before exiting. This will not save cookies that have expired or that have no expiry time (so-called "session cookies"), but also see −−keep−session−cookies. 576 | 577 | −−keep−session−cookies 578 | 579 | When specified, causes −−save−cookies to also save session cookies. Session cookies are normally not saved because they are meant to be kept in memory and forgotten when you exit the browser. Saving them is useful on sites that require you to log in or to visit the home page before you can access some pages. With this option, multiple Wget runs are considered a single browser session as far as the site is concerned. 580 | 581 | Since the cookie file format does not normally carry session cookies, Wget marks them with an expiry timestamp of 0. Wget’s −−load−cookies recognizes those as session cookies, but it might confuse other browsers. Also note that cookies so loaded will be treated as other session cookies, which means that if you want −−save−cookies to preserve them again, you must use −−keep−session−cookies again. 582 | 583 | −−ignore−length 584 | 585 | Unfortunately, some HTTP servers ( CGI programs, to be more precise) send out bogus "Content−Length" headers, which makes Wget go wild, as it thinks not all the document was retrieved. You can spot this syndrome if Wget retries getting the same document again and again, each time claiming that the (otherwise normal) connection has closed on the very same byte. 586 | 587 | With this option, Wget will ignore the "Content−Length" header−−−as if it never existed. 588 | 589 | −−header=header-line 590 | 591 | Send header-line along with the rest of the headers in each HTTP request. The supplied header is sent as-is, which means it must contain name and value separated by colon, and must not contain newlines. 592 | 593 | You may define more than one additional header by specifying −−header more than once. 594 | 595 | wget −−header='Accept−Charset: iso−8859−2' \ 596 | −−header='Accept−Language: hr' \ 597 | http://fly.srk.fer.hr/ 598 | Specification of an empty string as the header value will clear all previous user-defined headers. 599 | 600 | As of Wget 1.10, this option can be used to override headers otherwise generated automatically. This example instructs Wget to connect to localhost, but to specify foo.bar in the "Host" header: 601 | 602 | wget −−header="Host: foo.bar" http://localhost/ 603 | In versions of Wget prior to 1.10 such use of −−header caused sending of duplicate headers. 604 | 605 | −−compression=type 606 | 607 | Choose the type of compression to be used. Legal values are auto, gzip and none. 608 | 609 | If auto or gzip are specified, Wget asks the server to compress the file using the gzip compression format. If the server compresses the file and responds with the "Content−Encoding" header field set appropriately, the file will be decompressed automatically. 610 | 611 | If none is specified, wget will not ask the server to compress the file and will not decompress any server responses. This is the default. 612 | 613 | Compression support is currently experimental. In case it is turned on, please report any bugs to "bug−wget@gnu.org". 614 | 615 | −−max−redirect=number 616 | 617 | Specifies the maximum number of redirections to follow for a resource. The default is 20, which is usually far more than necessary. However, on those occasions where you want to allow more (or fewer), this is the option to use. 618 | 619 | −−proxy−user=user 620 | −−proxy−password=password 621 | 622 | Specify the username user and password password for authentication on a proxy server. Wget will encode them using the "basic" authentication scheme. 623 | 624 | Security considerations similar to those with −−http−password pertain here as well. 625 | 626 | −−referer=url 627 | 628 | Include ‘Referer: url’ header in HTTP request. Useful for retrieving documents with server-side processing that assume they are always being retrieved by interactive web browsers and only come out properly when Referer is set to one of the pages that point to them. 629 | 630 | −−save−headers 631 | 632 | Save the headers sent by the HTTP server to the file, preceding the actual contents, with an empty line as the separator. 633 | 634 | −U agent-string 635 | −−user−agent=agent-string 636 | 637 | Identify as agent-string to the HTTP server. 638 | 639 | The HTTP protocol allows the clients to identify themselves using a "User−Agent" header field. This enables distinguishing the WWW software, usually for statistical purposes or for tracing of protocol violations. Wget normally identifies as Wget/version, version being the current version number of Wget. 640 | 641 | However, some sites have been known to impose the policy of tailoring the output according to the "User−Agent"−supplied information. While this is not such a bad idea in theory, it has been abused by servers denying information to clients other than (historically) Netscape or, more frequently, Microsoft Internet Explorer. This option allows you to change the "User−Agent" line issued by Wget. Use of this option is discouraged, unless you really know what you are doing. 642 | 643 | Specifying empty user agent with −−user−agent="" instructs Wget not to send the "User−Agent" header in HTTP requests. 644 | 645 | −−post−data=string 646 | −−post−file=file 647 | 648 | Use POST as the method for all HTTP requests and send the specified data in the request body. −−post−data sends string as data, whereas −−post−file sends the contents of file. Other than that, they work in exactly the same way. In particular, they both expect content of the form "key1=value1&key2=value2", with percent-encoding for special characters; the only difference is that one expects its content as a command-line parameter and the other accepts its content from a file. In particular, −−post−file is not for transmitting files as form attachments: those must appear as "key=value" data (with appropriate percent-coding) just like everything else. Wget does not currently support "multipart/form−data" for transmitting POST data; only "application/x−www−form−urlencoded". Only one of −−post−data and −−post−file should be specified. 649 | 650 | Please note that wget does not require the content to be of the form "key1=value1&key2=value2", and neither does it test for it. Wget will simply transmit whatever data is provided to it. Most servers however expect the POST data to be in the above format when processing HTML Forms. 651 | 652 | When sending a POST request using the −−post−file option, Wget treats the file as a binary file and will send every character in the POST request without stripping trailing newline or formfeed characters. Any other control characters in the text will also be sent as-is in the POST request. 653 | 654 | Please be aware that Wget needs to know the size of the POST data in advance. Therefore the argument to "−−post−file" must be a regular file; specifying a FIFO or something like /dev/stdin won’t work. It’s not quite clear how to work around this limitation inherent in HTTP/1.0. Although HTTP/1.1 introduces chunked transfer that doesn’t require knowing the request length in advance, a client can’t use chunked unless it knows it’s talking to an HTTP/1.1 server. And it can’t know that until it receives a response, which in turn requires the request to have been completed -- a chicken-and-egg problem. 655 | 656 | Note: As of version 1.15 if Wget is redirected after the POST request is completed, its behaviour will depend on the response code returned by the server. In case of a 301 Moved Permanently, 302 Moved Temporarily or 307 Temporary Redirect, Wget will, in accordance with RFC2616, continue to send a POST request. In case a server wants the client to change the Request method upon redirection, it should send a 303 See Other response code. 657 | 658 | This example shows how to log in to a server using POST and then proceed to download the desired pages, presumably only accessible to authorized users: 659 | 660 | # Log in to the server. This can be done only once. 661 | wget −−save−cookies cookies.txt \ 662 | −−post−data 'user=foo&password=bar' \ 663 | http://example.com/auth.php 664 | # Now grab the page or pages we care about. 665 | wget −−load−cookies cookies.txt \ 666 | −p http://example.com/interesting/article.php 667 | If the server is using session cookies to track user authentication, the above will not work because −−save−cookies will not save them (and neither will browsers) and the cookies.txt file will be empty. In that case use −−keep−session−cookies along with −−save−cookies to force saving of session cookies. 668 | 669 | −−method=HTTP-Method 670 | 671 | For the purpose of RESTful scripting, Wget allows sending of other HTTP Methods without the need to explicitly set them using −−header=Header−Line. Wget will use whatever string is passed to it after −−method as the HTTP Method to the server. 672 | 673 | −−body−data=Data-String 674 | −−body−file=Data-File 675 | 676 | Must be set when additional data needs to be sent to the server along with the Method specified using −−method. −−body−data sends string as data, whereas −−body−file sends the contents of file. Other than that, they work in exactly the same way. 677 | 678 | Currently, −−body−file is not for transmitting files as a whole. Wget does not currently support "multipart/form−data" for transmitting data; only "application/x−www−form−urlencoded". In the future, this may be changed so that wget sends the −−body−file as a complete file instead of sending its contents to the server. Please be aware that Wget needs to know the contents of BODY Data in advance, and hence the argument to −−body−file should be a regular file. See −−post−file for a more detailed explanation. Only one of −−body−data and −−body−file should be specified. 679 | 680 | If Wget is redirected after the request is completed, Wget will suspend the current method and send a GET request till the redirection is completed. This is true for all redirection response codes except 307 Temporary Redirect which is used to explicitly specify that the request method should not change. Another exception is when the method is set to "POST", in which case the redirection rules specified under −−post−data are followed. 681 | 682 | −−content−disposition 683 | 684 | If this is set to on, experimental (not fully-functional) support for "Content−Disposition" headers is enabled. This can currently result in extra round-trips to the server for a "HEAD" request, and is known to suffer from a few bugs, which is why it is not currently enabled by default. 685 | 686 | This option is useful for some file-downloading CGI programs that use "Content−Disposition" headers to describe what the name of a downloaded file should be. 687 | 688 | When combined with −−metalink−over−http and −−trust−server−names, a Content-Type: application/metalink4+xml file is named using the "Content−Disposition" filename field, if available. 689 | 690 | −−content−on−error 691 | 692 | If this is set to on, wget will not skip the content when the server responds with a http status code that indicates error. 693 | 694 | −−trust−server−names 695 | 696 | If this is set, on a redirect, the local file name will be based on the redirection URL. By default the local file name is based on the original URL. When doing recursive retrieving this can be helpful because in many web sites redirected URLs correspond to an underlying file structure, while link URLs do not. 697 | 698 | −−auth−no−challenge 699 | 700 | If this option is given, Wget will send Basic HTTP authentication information (plaintext username and password) for all requests, just like Wget 1.10.2 and prior did by default. 701 | 702 | Use of this option is not recommended, and is intended only to support some few obscure servers, which never send HTTP authentication challenges, but accept unsolicited auth info, say, in addition to form-based authentication. 703 | 704 | −−retry−on−http−error=code[,code,...] 705 | 706 | Consider given HTTP response codes as non-fatal, transient errors. Supply a comma-separated list of 3−digit HTTP response codes as argument. Useful to work around special circumstances where retries are required, but the server responds with an error code normally not retried by Wget. Such errors might be 503 (Service Unavailable) and 429 (Too Many Requests). Retries enabled by this option are performed subject to the normal retry timing and retry count limitations of Wget. 707 | 708 | Using this option is intended to support special use cases only and is generally not recommended, as it can force retries even in cases where the server is actually trying to decrease its load. Please use wisely and only if you know what you are doing. 709 | 710 | HTTPS ( SSL/TLS ) Options 711 | To support encrypted HTTP ( HTTPS ) downloads, Wget must be compiled with an external SSL library. The current default is GnuTLS. In addition, Wget also supports HSTS ( HTTP Strict Transport Security). If Wget is compiled without SSL support, none of these options are available. 712 | −−secure−protocol=protocol 713 | 714 | Choose the secure protocol to be used. Legal values are auto, SSLv2, SSLv3, TLSv1, TLSv1_1, TLSv1_2 and PFS . If auto is used, the SSL library is given the liberty of choosing the appropriate protocol automatically, which is achieved by sending a TLSv1 greeting. This is the default. 715 | 716 | Specifying SSLv2, SSLv3, TLSv1, TLSv1_1 or TLSv1_2 forces the use of the corresponding protocol. This is useful when talking to old and buggy SSL server implementations that make it hard for the underlying SSL library to choose the correct protocol version. Fortunately, such servers are quite rare. 717 | 718 | Specifying PFS enforces the use of the so-called Perfect Forward Security cipher suites. In short, PFS adds security by creating a one-time key for each SSL connection. It has a bit more CPU impact on client and server. We use known to be secure ciphers (e.g. no MD4 ) and the TLS protocol. 719 | 720 | −−https−only 721 | 722 | When in recursive mode, only HTTPS links are followed. 723 | 724 | −−no−check−certificate 725 | 726 | Don’t check the server certificate against the available certificate authorities. Also don’t require the URL host name to match the common name presented by the certificate. 727 | 728 | As of Wget 1.10, the default is to verify the server’s certificate against the recognized certificate authorities, breaking the SSL handshake and aborting the download if the verification fails. Although this provides more secure downloads, it does break interoperability with some sites that worked with previous Wget versions, particularly those using self-signed, expired, or otherwise invalid certificates. This option forces an "insecure" mode of operation that turns the certificate verification errors into warnings and allows you to proceed. 729 | 730 | If you encounter "certificate verification" errors or ones saying that "common name doesn’t match requested host name", you can use this option to bypass the verification and proceed with the download. Only use this option if you are otherwise convinced of the site’s authenticity, or if you really don’t care about the validity of its certificate. It is almost always a bad idea not to check the certificates when transmitting confidential or important data. For self−signed/internal certificates, you should download the certificate and verify against that instead of forcing this insecure mode. If you are really sure of not desiring any certificate verification, you can specify −−check−certificate=quiet to tell wget to not print any warning about invalid certificates, albeit in most cases this is the wrong thing to do. 731 | 732 | −−certificate=file 733 | 734 | Use the client certificate stored in file. This is needed for servers that are configured to require certificates from the clients that connect to them. Normally a certificate is not required and this switch is optional. 735 | 736 | −−certificate−type=type 737 | 738 | Specify the type of the client certificate. Legal values are PEM (assumed by default) and DER , also known as ASN1 . 739 | 740 | −−private−key=file 741 | 742 | Read the private key from file. This allows you to provide the private key in a file separate from the certificate. 743 | 744 | −−private−key−type=type 745 | 746 | Specify the type of the private key. Accepted values are PEM (the default) and DER . 747 | 748 | −−ca−certificate=file 749 | 750 | Use file as the file with the bundle of certificate authorities (" CA" ) to verify the peers. The certificates must be in PEM format. 751 | 752 | Without this option Wget looks for CA certificates at the system-specified locations, chosen at OpenSSL installation time. 753 | 754 | −−ca−directory=directory 755 | 756 | Specifies directory containing CA certificates in PEM format. Each file contains one CA certificate, and the file name is based on a hash value derived from the certificate. This is achieved by processing a certificate directory with the "c_rehash" utility supplied with OpenSSL. Using −−ca−directory is more efficient than −−ca−certificate when many certificates are installed because it allows Wget to fetch certificates on demand. 757 | 758 | Without this option Wget looks for CA certificates at the system-specified locations, chosen at OpenSSL installation time. 759 | 760 | −−crl−file=file 761 | 762 | Specifies a CRL file in file. This is needed for certificates that have been revocated by the CAs. 763 | 764 | −−pinnedpubkey=file/hashes 765 | 766 | Tells wget to use the specified public key file (or hashes) to verify the peer. This can be a path to a file which contains a single public key in PEM or DER format, or any number of base64 encoded sha256 hashes preceded by "sha256//" and separated by ";" 767 | 768 | When negotiating a TLS or SSL connection, the server sends a certificate indicating its identity. A public key is extracted from this certificate and if it does not exactly match the public key(s) provided to this option, wget will abort the connection before sending or receiving any data. 769 | 770 | −−random−file=file 771 | 772 | [OpenSSL and LibreSSL only] Use file as the source of random data for seeding the pseudo-random number generator on systems without /dev/urandom. 773 | 774 | On such systems the SSL library needs an external source of randomness to initialize. Randomness may be provided by EGD (see −−egd−file below) or read from an external source specified by the user. If this option is not specified, Wget looks for random data in $RANDFILE or, if that is unset, in $HOME/.rnd. 775 | 776 | If you’re getting the "Could not seed OpenSSL PRNG ; disabling SSL." error, you should provide random data using some of the methods described above. 777 | 778 | −−egd−file=file 779 | 780 | [OpenSSL only] Use file as the EGD socket. EGD stands for Entropy Gathering Daemon, a user-space program that collects data from various unpredictable system sources and makes it available to other programs that might need it. Encryption software, such as the SSL library, needs sources of non-repeating randomness to seed the random number generator used to produce cryptographically strong keys. 781 | 782 | OpenSSL allows the user to specify his own source of entropy using the "RAND_FILE" environment variable. If this variable is unset, or if the specified file does not produce enough randomness, OpenSSL will read random data from EGD socket specified using this option. 783 | 784 | If this option is not specified (and the equivalent startup command is not used), EGD is never contacted. EGD is not needed on modern Unix systems that support /dev/urandom. 785 | 786 | −−no−hsts 787 | 788 | Wget supports HSTS ( HTTP Strict Transport Security, RFC 6797 ) by default. Use −−no−hsts to make Wget act as a non-HSTS-compliant UA. As a consequence, Wget would ignore all the "Strict−Transport−Security" headers, and would not enforce any existing HSTS policy. 789 | 790 | −−hsts−file=file 791 | 792 | By default, Wget stores its HSTS database in ~/.wget−hsts. You can use −−hsts−file to override this. Wget will use the supplied file as the HSTS database. Such file must conform to the correct HSTS database format used by Wget. If Wget cannot parse the provided file, the behaviour is unspecified. 793 | 794 | The Wget’s HSTS database is a plain text file. Each line contains an HSTS entry (ie. a site that has issued a "Strict−Transport−Security" header and that therefore has specified a concrete HSTS policy to be applied). Lines starting with a dash ("#") are ignored by Wget. Please note that in spite of this convenient human-readability hand-hacking the HSTS database is generally not a good idea. 795 | 796 | An HSTS entry line consists of several fields separated by one or more whitespace: 797 | 798 | " SP [] SP SP SP " 799 | 800 | The hostname and port fields indicate the hostname and port to which the given HSTS policy applies. The port field may be zero, and it will, in most of the cases. That means that the port number will not be taken into account when deciding whether such HSTS policy should be applied on a given request (only the hostname will be evaluated). When port is different to zero, both the target hostname and the port will be evaluated and the HSTS policy will only be applied if both of them match. This feature has been included for testing/development purposes only. The Wget testsuite (in testenv/) creates HSTS databases with explicit ports with the purpose of ensuring Wget’s correct behaviour. Applying HSTS policies to ports other than the default ones is discouraged by RFC 6797 (see Appendix B "Differences between HSTS Policy and Same-Origin Policy"). Thus, this functionality should not be used in production environments and port will typically be zero. The last three fields do what they are expected to. The field include_subdomains can either be 1 or 0 and it signals whether the subdomains of the target domain should be part of the given HSTS policy as well. The created and max-age fields hold the timestamp values of when such entry was created (first seen by Wget) and the HSTS-defined value ’max−age’, which states how long should that HSTS policy remain active, measured in seconds elapsed since the timestamp stored in created. Once that time has passed, that HSTS policy will no longer be valid and will eventually be removed from the database. 801 | 802 | If you supply your own HSTS database via −−hsts−file, be aware that Wget may modify the provided file if any change occurs between the HSTS policies requested by the remote servers and those in the file. When Wget exists, it effectively updates the HSTS database by rewriting the database file with the new entries. 803 | 804 | If the supplied file does not exist, Wget will create one. This file will contain the new HSTS entries. If no HSTS entries were generated (no "Strict−Transport−Security" headers were sent by any of the servers) then no file will be created, not even an empty one. This behaviour applies to the default database file (~/.wget−hsts) as well: it will not be created until some server enforces an HSTS policy. 805 | 806 | Care is taken not to override possible changes made by other Wget processes at the same time over the HSTS database. Before dumping the updated HSTS entries on the file, Wget will re-read it and merge the changes. 807 | 808 | Using a custom HSTS database and/or modifying an existing one is discouraged. For more information about the potential security threats arised from such practice, see section 14 "Security Considerations" of RFC 6797, specially section 14.9 "Creative Manipulation of HSTS Policy Store". 809 | 810 | −−warc−file=file 811 | 812 | Use file as the destination WARC file. 813 | 814 | −−warc−header=string 815 | 816 | Use string into as the warcinfo record. 817 | 818 | −−warc−max−size=size 819 | 820 | Set the maximum size of the WARC files to size. 821 | 822 | −−warc−cdx 823 | 824 | Write CDX index files. 825 | 826 | −−warc−dedup=file 827 | 828 | Do not store records listed in this CDX file. 829 | 830 | −−no−warc−compression 831 | 832 | Do not compress WARC files with GZIP. 833 | 834 | −−no−warc−digests 835 | 836 | Do not calculate SHA1 digests. 837 | 838 | −−no−warc−keep−log 839 | 840 | Do not store the log file in a WARC record. 841 | 842 | −−warc−tempdir=dir 843 | 844 | Specify the location for temporary files created by the WARC writer. 845 | 846 | FTP Options 847 | −−ftp−user=user 848 | −−ftp−password=password 849 | 850 | Specify the username user and password password on an FTP server. Without this, or the corresponding startup option, the password defaults to −wget@, normally used for anonymous FTP. 851 | 852 | Another way to specify username and password is in the URL itself. Either method reveals your password to anyone who bothers to run "ps". To prevent the passwords from being seen, store them in .wgetrc or .netrc, and make sure to protect those files from other users with "chmod". If the passwords are really important, do not leave them lying in those files either−−−edit the files and delete them after Wget has started the download. 853 | 854 | −−no−remove−listing 855 | 856 | Don’t remove the temporary .listing files generated by FTP retrievals. Normally, these files contain the raw directory listings received from FTP servers. Not removing them can be useful for debugging purposes, or when you want to be able to easily check on the contents of remote server directories (e.g. to verify that a mirror you’re running is complete). 857 | 858 | Note that even though Wget writes to a known filename for this file, this is not a security hole in the scenario of a user making .listing a symbolic link to /etc/passwd or something and asking "root" to run Wget in his or her directory. Depending on the options used, either Wget will refuse to write to .listing, making the globbing/recursion/time−stamping operation fail, or the symbolic link will be deleted and replaced with the actual .listing file, or the listing will be written to a .listing.number file. 859 | 860 | Even though this situation isn’t a problem, though, "root" should never run Wget in a non-trusted user’s directory. A user could do something as simple as linking index.html to /etc/passwd and asking "root" to run Wget with −N or −r so the file will be overwritten. 861 | 862 | −−no−glob 863 | 864 | Turn off FTP globbing. Globbing refers to the use of shell-like special characters (wildcards), like *, ?, [ and ] to retrieve more than one file from the same directory at once, like: 865 | 866 | wget ftp://gnjilux.srk.fer.hr/*.msg 867 | By default, globbing will be turned on if the URL contains a globbing character. This option may be used to turn globbing on or off permanently. 868 | 869 | You may have to quote the URL to protect it from being expanded by your shell. Globbing makes Wget look for a directory listing, which is system-specific. This is why it currently works only with Unix FTP servers (and the ones emulating Unix "ls" output). 870 | 871 | −−no−passive−ftp 872 | 873 | Disable the use of the passive FTP transfer mode. Passive FTP mandates that the client connect to the server to establish the data connection rather than the other way around. 874 | 875 | If the machine is connected to the Internet directly, both passive and active FTP should work equally well. Behind most firewall and NAT configurations passive FTP has a better chance of working. However, in some rare firewall configurations, active FTP actually works when passive FTP doesn’t. If you suspect this to be the case, use this option, or set "passive_ftp=off" in your init file. 876 | 877 | −−preserve−permissions 878 | 879 | Preserve remote file permissions instead of permissions set by umask. 880 | 881 | −−retr−symlinks 882 | 883 | By default, when retrieving FTP directories recursively and a symbolic link is encountered, the symbolic link is traversed and the pointed-to files are retrieved. Currently, Wget does not traverse symbolic links to directories to download them recursively, though this feature may be added in the future. 884 | 885 | When −−retr−symlinks=no is specified, the linked-to file is not downloaded. Instead, a matching symbolic link is created on the local filesystem. The pointed-to file will not be retrieved unless this recursive retrieval would have encountered it separately and downloaded it anyway. This option poses a security risk where a malicious FTP Server may cause Wget to write to files outside of the intended directories through a specially crafted .LISTING file. 886 | 887 | Note that when retrieving a file (not a directory) because it was specified on the command-line, rather than because it was recursed to, this option has no effect. Symbolic links are always traversed in this case. 888 | 889 | FTPS Options 890 | −−ftps−implicit 891 | 892 | This option tells Wget to use FTPS implicitly. Implicit FTPS consists of initializing SSL/TLS from the very beginning of the control connection. This option does not send an "AUTH TLS" command: it assumes the server speaks FTPS and directly starts an SSL/TLS connection. If the attempt is successful, the session continues just like regular FTPS ("PBSZ" and "PROT" are sent, etc.). Implicit FTPS is no longer a requirement for FTPS implementations, and thus many servers may not support it. If −−ftps−implicit is passed and no explicit port number specified, the default port for implicit FTPS, 990, will be used, instead of the default port for the "normal" (explicit) FTPS which is the same as that of FTP, 21. 893 | 894 | −−no−ftps−resume−ssl 895 | 896 | Do not resume the SSL/TLS session in the data channel. When starting a data connection, Wget tries to resume the SSL/TLS session previously started in the control connection. SSL/TLS session resumption avoids performing an entirely new handshake by reusing the SSL/TLS parameters of a previous session. Typically, the FTPS servers want it that way, so Wget does this by default. Under rare circumstances however, one might want to start an entirely new SSL/TLS session in every data connection. This is what −−no−ftps−resume−ssl is for. 897 | 898 | −−ftps−clear−data−connection 899 | 900 | All the data connections will be in plain text. Only the control connection will be under SSL/TLS. Wget will send a "PROT C" command to achieve this, which must be approved by the server. 901 | 902 | −−ftps−fallback−to−ftp 903 | 904 | Fall back to FTP if FTPS is not supported by the target server. For security reasons, this option is not asserted by default. The default behaviour is to exit with an error. If a server does not successfully reply to the initial "AUTH TLS" command, or in the case of implicit FTPS, if the initial SSL/TLS connection attempt is rejected, it is considered that such server does not support FTPS. 905 | 906 | Recursive Retrieval Options 907 | 908 | −r 909 | 910 | −−recursive 911 | 912 | Turn on recursive retrieving. The default maximum depth is 5. 913 | 914 | −l depth 915 | −−level=depth 916 | 917 | Specify recursion maximum depth level depth. 918 | 919 | −−delete−after 920 | 921 | This option tells Wget to delete every single file it downloads, after having done so. It is useful for pre-fetching popular pages through a proxy, e.g.: 922 | 923 | wget −r −nd −−delete−after http://whatever.com/~popular/page/ 924 | The −r option is to retrieve recursively, and −nd to not create directories. 925 | 926 | Note that −−delete−after deletes files on the local machine. It does not issue the DELE command to remote FTP sites, for instance. Also note that when −−delete−after is specified, −−convert−links is ignored, so .orig files are simply not created in the first place. 927 | 928 | −k 929 | 930 | −−convert−links 931 | 932 | After the download is complete, convert the links in the document to make them suitable for local viewing. This affects not only the visible hyperlinks, but any part of the document that links to external content, such as embedded images, links to style sheets, hyperlinks to non-HTML content, etc. 933 | 934 | Each link will be changed in one of the two ways: 935 | 936 | • 937 | 938 | The links to files that have been downloaded by Wget will be changed to refer to the file they point to as a relative link. 939 | 940 | Example: if the downloaded file /foo/doc.html links to /bar/img.gif, also downloaded, then the link in doc.html will be modified to point to ../bar/img.gif. This kind of transformation works reliably for arbitrary combinations of directories. 941 | 942 | • 943 | 944 | The links to files that have not been downloaded by Wget will be changed to include host name and absolute path of the location they point to. 945 | 946 | Example: if the downloaded file /foo/doc.html links to /bar/img.gif (or to ../bar/img.gif), then the link in doc.html will be modified to point to http://hostname/bar/img.gif. 947 | 948 | Because of this, local browsing works reliably: if a linked file was downloaded, the link will refer to its local name; if it was not downloaded, the link will refer to its full Internet address rather than presenting a broken link. The fact that the former links are converted to relative links ensures that you can move the downloaded hierarchy to another directory. 949 | 950 | Note that only at the end of the download can Wget know which links have been downloaded. Because of that, the work done by −k will be performed at the end of all the downloads. 951 | 952 | −−convert−file−only 953 | 954 | This option converts only the filename part of the URLs, leaving the rest of the URLs untouched. This filename part is sometimes referred to as the "basename", although we avoid that term here in order not to cause confusion. 955 | 956 | It works particularly well in conjunction with −−adjust−extension, although this coupling is not enforced. It proves useful to populate Internet caches with files downloaded from different hosts. 957 | 958 | Example: if some link points to //foo.com/bar.cgi?xyz with −−adjust−extension asserted and its local destination is intended to be ./foo.com/bar.cgi?xyz.css, then the link would be converted to //foo.com/bar.cgi?xyz.css. Note that only the filename part has been modified. The rest of the URL has been left untouched, including the net path ("//") which would otherwise be processed by Wget and converted to the effective scheme (ie. "http://"). 959 | 960 | −K 961 | 962 | −−backup−converted 963 | 964 | When converting a file, back up the original version with a .orig suffix. Affects the behavior of −N. 965 | 966 | −m 967 | 968 | −−mirror 969 | 970 | Turn on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. It is currently equivalent to −r −N −l inf −−no−remove−listing. 971 | 972 | −p 973 | 974 | −−page−requisites 975 | 976 | This option causes Wget to download all the files that are necessary to properly display a given HTML page. This includes such things as inlined images, sounds, and referenced stylesheets. 977 | 978 | Ordinarily, when downloading a single HTML page, any requisite documents that may be needed to display it properly are not downloaded. Using −r together with −l can help, but since Wget does not ordinarily distinguish between external and inlined documents, one is generally left with "leaf documents" that are missing their requisites. 979 | 980 | For instance, say document 1.html contains an "" tag referencing 1.gif and an "" tag pointing to external document 2.html. Say that 2.html is similar but that its image is 2.gif and it links to 3.html. Say this continues up to some arbitrarily high number. 981 | 982 | If one executes the command: 983 | 984 | wget −r −l 2 http:///1.html 985 | then 1.html, 1.gif, 2.html, 2.gif, and 3.html will be downloaded. As you can see, 3.html is without its requisite 3.gif because Wget is simply counting the number of hops (up to 2) away from 1.html in order to determine where to stop the recursion. However, with this command: 986 | 987 | wget −r −l 2 −p http:///1.html 988 | all the above files and 3.html’s requisite 3.gif will be downloaded. Similarly, 989 | 990 | wget −r −l 1 −p http:///1.html 991 | will cause 1.html, 1.gif, 2.html, and 2.gif to be downloaded. One might think that: 992 | 993 | wget −r −l 0 −p http:///1.html 994 | would download just 1.html and 1.gif, but unfortunately this is not the case, because −l 0 is equivalent to −l inf−−−that is, infinite recursion. To download a single HTML page (or a handful of them, all specified on the command-line or in a −i URL input file) and its (or their) requisites, simply leave off −r and −l: 995 | 996 | wget −p http:///1.html 997 | Note that Wget will behave as if −r had been specified, but only that single page and its requisites will be downloaded. Links from that page to external documents will not be followed. Actually, to download a single page and all its requisites (even if they exist on separate websites), and make sure the lot displays properly locally, this author likes to use a few options in addition to −p: 998 | 999 | wget −E −H −k −K −p http:/// 1000 | To finish off this topic, it’s worth knowing that Wget’s idea of an external document link is any URL specified in an "" tag, an "" tag, or a "" tag other than "". 1001 | 1002 | −−strict−comments 1003 | 1004 | Turn on strict parsing of HTML comments. The default is to terminate comments at the first occurrence of −−>. 1005 | 1006 | According to specifications, HTML comments are expressed as SGML declarations. Declaration is special markup that begins with , such as , that may contain comments between a pair of −− delimiters. HTML comments are "empty declarations", SGML declarations without any non-comment text. Therefore, is a valid comment, and so is , but is not. 1007 | 1008 | On the other hand, most HTML writers don’t perceive comments as anything other than text delimited with , which is not quite the same. For example, something like works as a valid comment as long as the number of dashes is a multiple of four (!). If not, the comment technically lasts until the next −−, which may be at the other end of the document. Because of this, many popular browsers completely ignore the specification and implement what users have come to expect: comments delimited with . 1009 | 1010 | Until version 1.9, Wget interpreted comments strictly, which resulted in missing links in many web pages that displayed fine in browsers, but had the misfortune of containing non-compliant comments. Beginning with version 1.9, Wget has joined the ranks of clients that implements "naive" comments, terminating each comment at the first occurrence of −−>. 1011 | 1012 | If, for whatever reason, you want strict comment parsing, use this option to turn it on. 1013 | 1014 | Recursive Accept/Reject Options 1015 | −A acclist −−accept acclist 1016 | −R rejlist −−reject rejlist 1017 | 1018 | Specify comma-separated lists of file name suffixes or patterns to accept or reject. Note that if any of the wildcard characters, *, ?, [ or ], appear in an element of acclist or rejlist, it will be treated as a pattern, rather than a suffix. In this case, you have to enclose the pattern into quotes to prevent your shell from expanding it, like in −A "*.mp3" or −A ’*.mp3’. 1019 | 1020 | −−accept−regex urlregex 1021 | −−reject−regex urlregex 1022 | 1023 | Specify a regular expression to accept or reject the complete URL. 1024 | 1025 | −−regex−type regextype 1026 | 1027 | Specify the regular expression type. Possible types are posix or pcre. Note that to be able to use pcre type, wget has to be compiled with libpcre support. 1028 | 1029 | −D domain-list 1030 | −−domains=domain-list 1031 | 1032 | Set domains to be followed. domain-list is a comma-separated list of domains. Note that it does not turn on −H. 1033 | 1034 | −−exclude−domains domain-list 1035 | 1036 | Specify the domains that are not to be followed. 1037 | 1038 | −−follow−ftp 1039 | 1040 | Follow FTP links from HTML documents. Without this option, Wget will ignore all the FTP links. 1041 | 1042 | −−follow−tags=list 1043 | 1044 | Wget has an internal table of HTML tag / attribute pairs that it considers when looking for linked documents during a recursive retrieval. If a user wants only a subset of those tags to be considered, however, he or she should be specify such tags in a comma-separated list with this option. 1045 | 1046 | −−ignore−tags=list 1047 | 1048 | This is the opposite of the −−follow−tags option. To skip certain HTML tags when recursively looking for documents to download, specify them in a comma-separated list. 1049 | 1050 | In the past, this option was the best bet for downloading a single page and its requisites, using a command-line like: 1051 | 1052 | wget −−ignore−tags=a,area −H −k −K −r http:/// 1053 | However, the author of this option came across a page with tags like "" and came to the realization that specifying tags to ignore was not enough. One can’t just tell Wget to ignore "", because then stylesheets will not be downloaded. Now the best bet for downloading a single page and its requisites is the dedicated −−page−requisites option. 1054 | 1055 | −−ignore−case 1056 | 1057 | Ignore case when matching files and directories. This influences the behavior of −R, −A, −I, and −X options, as well as globbing implemented when downloading from FTP sites. For example, with this option, −A "*.txt" will match file1.txt, but also file2.TXT, file3.TxT, and so on. The quotes in the example are to prevent the shell from expanding the pattern. 1058 | 1059 | −H 1060 | 1061 | −−span−hosts 1062 | 1063 | Enable spanning across hosts when doing recursive retrieving. 1064 | 1065 | −L 1066 | 1067 | −−relative 1068 | 1069 | Follow relative links only. Useful for retrieving a specific home page without any distractions, not even those from the same hosts. 1070 | 1071 | −I list 1072 | −−include−directories=list 1073 | 1074 | Specify a comma-separated list of directories you wish to follow when downloading. Elements of list may contain wildcards. 1075 | 1076 | −X list 1077 | −−exclude−directories=list 1078 | 1079 | Specify a comma-separated list of directories you wish to exclude from download. Elements of list may contain wildcards. 1080 | 1081 | −np 1082 | 1083 | −−no−parent 1084 | 1085 | Do not ever ascend to the parent directory when retrieving recursively. This is a useful option, since it guarantees that only the files below a certain hierarchy will be downloaded. 1086 | 1087 | ENVIRONMENT 1088 | Wget supports proxies for both HTTP and FTP retrievals. The standard way to specify proxy location, which Wget recognizes, is using the following environment variables: 1089 | http_proxy 1090 | https_proxy 1091 | 1092 | If set, the http_proxy and https_proxy variables should contain the URLs of the proxies for HTTP and HTTPS connections respectively. 1093 | 1094 | ftp_proxy 1095 | 1096 | This variable should contain the URL of the proxy for FTP connections. It is quite common that http_proxy and ftp_proxy are set to the same URL. 1097 | 1098 | no_proxy 1099 | 1100 | This variable should contain a comma-separated list of domain extensions proxy should not be used for. For instance, if the value of no_proxy is .mit.edu, proxy will not be used to retrieve documents from MIT. 1101 | 1102 | EXIT STATUS 1103 | Wget may return one of several error codes if it encounters problems. 1104 | 1105 | 0 1106 | 1107 | No problems occurred. 1108 | 1109 | 1 1110 | 1111 | Generic error code. 1112 | 1113 | 2 1114 | 1115 | Parse error−−−for instance, when parsing command-line options, the .wgetrc or .netrc... 1116 | 1117 | 3 1118 | 1119 | File I/O error. 1120 | 1121 | 4 1122 | 1123 | Network failure. 1124 | 1125 | 5 1126 | 1127 | SSL verification failure. 1128 | 1129 | 6 1130 | 1131 | Username/password authentication failure. 1132 | 1133 | 7 1134 | 1135 | Protocol errors. 1136 | 1137 | 8 1138 | 1139 | Server issued an error response. 1140 | 1141 | With the exceptions of 0 and 1, the lower-numbered exit codes take precedence over higher-numbered ones, when multiple types of errors are encountered. 1142 | 1143 | In versions of Wget prior to 1.12, Wget’s exit status tended to be unhelpful and inconsistent. Recursive downloads would virtually always return 0 (success), regardless of any issues encountered, and non-recursive fetches only returned the status corresponding to the most recently-attempted download. 1144 | 1145 | FILES 1146 | /usr/local/etc/wgetrc 1147 | 1148 | Default location of the global startup file. 1149 | 1150 | .wgetrc 1151 | 1152 | User startup file. 1153 | 1154 | BUGS 1155 | You are welcome to submit bug reports via the GNU Wget bug tracker (see ). 1156 | 1157 | Before actually submitting a bug report, please try to follow a few simple guidelines. 1158 | 1159 | 1. 1160 | 1161 | Please try to ascertain that the behavior you see really is a bug. If Wget crashes, it’s a bug. If Wget does not behave as documented, it’s a bug. If things work strange, but you are not sure about the way they are supposed to work, it might well be a bug, but you might want to double-check the documentation and the mailing lists. 1162 | 1163 | 2. 1164 | 1165 | Try to repeat the bug in as simple circumstances as possible. E.g. if Wget crashes while downloading wget −rl0 −kKE −t5 −−no−proxy http://example.com −o /tmp/log, you should try to see if the crash is repeatable, and if will occur with a simpler set of options. You might even try to start the download at the page where the crash occurred to see if that page somehow triggered the crash. 1166 | 1167 | Also, while I will probably be interested to know the contents of your .wgetrc file, just dumping it into the debug message is probably a bad idea. Instead, you should first try to see if the bug repeats with .wgetrc moved out of the way. Only if it turns out that .wgetrc settings affect the bug, mail me the relevant parts of the file. 1168 | 1169 | 3. 1170 | 1171 | Please start Wget with −d option and send us the resulting output (or relevant parts thereof). If Wget was compiled without debug support, recompile it−−−it is much easier to trace bugs with debug support on. 1172 | 1173 | Note: please make sure to remove any potentially sensitive information from the debug log before sending it to the bug address. The "−d" won’t go out of its way to collect sensitive information, but the log will contain a fairly complete transcript of Wget’s communication with the server, which may include passwords and pieces of downloaded data. Since the bug address is publically archived, you may assume that all bug reports are visible to the public. 1174 | 1175 | 4. 1176 | 1177 | If Wget has crashed, try to run it in a debugger, e.g. "gdb `which wget` core" and type "where" to get the backtrace. This may not work if the system administrator has disabled core files, but it is safe to try. 1178 | 1179 | SEE ALSO 1180 | This is not the complete manual for GNU Wget. For more complete information, including more detailed explanations of some of the options, and a number of commands available for use with .wgetrc files and the −e option, see the GNU Info entry for wget. 1181 | 1182 | AUTHOR 1183 | Originally written by Hrvoje NikÅ¡iÄ . 1184 | 1185 | COPYRIGHT 1186 | Copyright (c) 1996−2011, 2015, 2018 Free Software Foundation, Inc. 1187 | 1188 | Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled " GNU Free Documentation License". 1189 | --------------------------------------------------------------------------------