├── LICENSE ├── README.md └── libg /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Aditya Dutta 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # libg-fzf 2 | 3 | **Library Genesis TUI** using **fzf**. 4 | Using this, you can search the library genesis site (https://libgen.is or https://libgen.rs or https://libgen.st), filter your book out using `fzf` as well as download the selected book by just pressing `Enter`; everything right from your terminal! 5 | 6 | ## Usage 7 | `libg [OPTIONS] ` 8 | 9 | ### Options 10 | ``` 11 | OPTIONS Description Allowed Values 12 | ------- ----------- -------------- 13 | -b Search By author, title, publisher, year, isbn, language, md5, tags, extension. 14 | -n Number of Search Results per Page 25, 50, 100. 15 | -d Depth (Number of result pages to scan) Any positive integer. 16 | -s Sort Results By id, author, title, publisher, year, pages, language, filesize, extension. 17 | -r Reverse/Desecending Order 18 | ``` 19 | 20 | ## Config file 21 | This file is created automatically with default values after first use. 22 | Edit `$HOME/.config/libg/libg.sh` to change default values: 23 | 24 | ```bash 25 | DEFAULT_RESPERPAGE=100 # default number of search results per page (allowed values: 25, 50, 100) 26 | DEFAULT_DEPTH=1 # default number of result pages to scan (allowed values: any positive integer) 27 | LIBGEN_MIRROR="https://libgen.rs" # alternatives: *.is, *.rs, *.st 28 | DOWNLOAD_LOCATION="" # default download location (if empty then books will be downloaded to current directory) 29 | ``` 30 | 31 | **Note**: Giving any value except for 25, 50 or 100 for `DEFAULT_RESPERPAGE` will default it to 25. 32 | 33 | ## Installation 34 | Since it is just a small shell script, just download the script, give it executable permissions and place it in a directory that is in `$PATH`. 35 | 36 | ## Dependency 37 | The only dependency is `fzf`: https://github.com/junegunn/fzf. 38 | This is usually available on standard linux repositories for almost all distributions. 39 | The other dependencies are `sed`(GNU), `awk`, `curl` and `wget` but are present by default in most Linux installs. 40 | -------------------------------------------------------------------------------- /libg: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | CONFIG_DIR="$HOME/.config/libg" 4 | CONFIG_FILE="$CONFIG_DIR/libg.sh" 5 | if [ -f "$CONFIG_FILE" ]; then 6 | . "$CONFIG_FILE" 7 | else 8 | mkdir -p "$CONFIG_DIR" >/dev/null 9 | printf "DEFAULT_RESPERPAGE=100 # default number of search results per page (allowed values: 25, 50, 100)\ 10 | \nDEFAULT_DEPTH=1 # default number of result pages to scan (allowed values: any positive integer)\ 11 | \nLIBGEN_MIRROR=\"https://libgen.is\" # alternatives: *.is, *.rs, *.st\ 12 | \nDOWNLOAD_LOCATION=\"\" # default download location (if empty then books will be downloaded to current directory)" >"$CONFIG_FILE" 13 | . "$CONFIG_FILE" 14 | fi 15 | 16 | [ -z "$DOWNLOAD_LOCATION" ] || [ -d "$DOWNLOAD_LOCATION" ] && cd "$DOWNLOAD_LOCATION" 17 | 18 | basic_help() { 19 | printf "libg: Shell Script to scrape library genesis website and download books right from the terminal.\ 20 | \nUSAGE: libg [OPTIONS] \ 21 | \n\nOPTIONS Description Allowed Values\ 22 | \n------- ----------- --------------\ 23 | \n-b Search By author, title, publisher, year, isbn, language, md5, tags, extension.\ 24 | \n-n Number of Search Results per Page 25, 50, 100.\ 25 | \n-d Depth (Number of result pages to scan) Any positive integer.\ 26 | \n-s Sort Results By id, author, title, publisher, year, pages, language, filesize, extension.\ 27 | \n-r Reverse/Desecending Order\ 28 | \n\nCurrent Defaults (Can be changed by editing the script)\ 29 | \n----------------\ 30 | \nNumber of Search Results per Page $DEFAULT_RESPERPAGE \nDepth $DEFAULT_DEPTH \nLibgen Mirror $LIBGEN_MIRROR\n" 31 | } 32 | 33 | # -- PARSE OPTIONS -- 34 | while getopts b:n:d:s:rh opt; do 35 | case "$opt" in 36 | # SEARCHBY: author, title, publisher, year, isbn, language, md5, tags, extension 37 | b) SEARCHBY="$OPTARG" ;; 38 | # RESPERPAGE: 25, 50, 100 39 | n) RESPERPAGE="$OPTARG" ;; 40 | # DEPTH: Any positive integer 41 | d) DEPTH="$OPTARG" ;; 42 | # SORTBY: id, author, title, publisher, year, pages, language, filesize, extension 43 | s) SORTBY="$OPTARG" ;; 44 | # SORTORDER: ASC, DESC 45 | r) SORTORDER=DESC ;; 46 | h) basic_help && exit 0 ;; 47 | \?) basic_help && exit 1 ;; 48 | esac 49 | done 50 | shift $((OPTIND - 1)) 51 | 52 | # -- CHECK VALIDITY OF OPTIONS -- 53 | case "$SEARCHBY" in 54 | "") ;; author) ;; title) ;; publisher) ;; year) ;; isbn) ;; language) ;; md5) ;; tags) ;; extension) ;; 55 | *) printf "\033[31mERR\033[0m: Invalid Value for -b option!\nAllowed values: author, title, publisher, year, isbn, language, md5, tags, extension.\n" && exit 1 ;; 56 | esac 57 | 58 | case "$RESPERPAGE" in 59 | "") RESPERPAGE="$DEFAULT_RESPERPAGE" ;; 60 | 0) printf "\033[31mERR\033[0m: Invalid Value for -n option!\nAllowed values: 25, 50, 100.\n" && exit 1 ;; 61 | [0-9]*) ;; 62 | *) printf "\033[31mERR\033[0m: Invalid Value for -n option!\nAllowed values: 25, 50, 100.\n" && exit 1 ;; 63 | esac 64 | 65 | case "$DEPTH" in 66 | "") DEPTH="$DEFAULT_DEPTH" ;; 67 | 0) printf "\033[31mERR\033[0m: Invalid Value for -d option!\nAllowed values: Any positive integer.\n" && exit 1 ;; 68 | [0-9]*) ;; 69 | *) printf "\033[31mERR\033[0m: Invalid Value for -d option!\nAllowed values: Any positive integer.\n" && exit 1 ;; 70 | esac 71 | 72 | case "$SORTBY" in 73 | "") ;; id) ;; author) ;; title) ;; publisher) ;; year) ;; pages) ;; language) ;; filesize) ;; extension) ;; 74 | *) printf "\033[31mERR\033[0m: Invalid Value for -s option!\nAllowed values: id, author, title, publisher, year, pages, language, filesize, extension.\n" && exit 1 ;; 75 | esac 76 | 77 | case "$SORTORDER" in 78 | "") ;; ASC) ;; DESC) ;; 79 | *) printf "\033[31mERR\033[0m: Invalid value given for SORTORDER." && exit 1 ;; 80 | esac 81 | 82 | # -- GET SEARCH TERMS -- 83 | QUERY="$*" 84 | if [ -z "$QUERY" ]; then 85 | printf "Enter query: " 86 | read -r QUERY 87 | fi 88 | 89 | # -- SCRAPING -- 90 | page=1 91 | original_page="$(mktemp)" 92 | data_file="$(mktemp)" 93 | while [ "$page" -le "$DEPTH" ]; do 94 | printf "\033[1mGetting Page \033[35m$page... \033[0m" 95 | 96 | curl -s -G "$LIBGEN_MIRROR/search.php" \ 97 | --data-urlencode "req=$QUERY" \ 98 | --data-urlencode "column=$SEARCHBY" \ 99 | --data-urlencode "res=$RESPERPAGE" \ 100 | --data-urlencode "page=$page" \ 101 | --data-urlencode "sort=$SORTBY" \ 102 | --data-urlencode "sortmode=$SORTORDER" >"$original_page" 103 | status="$?" 104 | [ "$status" -eq 6 ] && printf "\033[31mFailed!\nERR\033[0m: Couldn't connect to Library Genesis! Check you internet connection and try again!\n" && exit 2 105 | 106 | printf "\033[32mCompleted!\033[0m\n" 107 | printf "\033[1mScraping Page \033[35m$page... \033[0m" 108 | if grep -q 'Edit' "$original_page"; then 109 | printf "\033[1mEnd of Results! Nothing on page \033[35m$page!\033[0m\n" 110 | [ "$page" -eq 1 ] && exit 111 | break 112 | fi 113 | # Extract the relevant part. Note that we are making only one pass through the file :) 114 | sed -i -n '/^Edit.*/,$p;/^<\/tr><\/table>$/q' "$original_page" 115 | # -- Cleaning up -- 116 | # Remove unnecessary part from first line and delete last line 117 | sed -i '1s_.* tags 119 | sed '/^\t*<\/tr>$\|^$/d' "$original_page" | 120 | # Remove opening and closing tags along with \r's 121 | sed 's_\t*]*>\r\?__g' | 122 | # -- Extracting authors -- 123 | sed 's_]*author["'\'']>\([^<]*\)_\1_g' | 124 | # -- Extracting bookname -- 125 | # Remove and tags first. 126 | # Now a author line can consist of three things: series, bookname, ISBNs separated by commas; and all of them separated by
tags. 127 | # Bookname is a must but others may or may not be there. So, convert the
tags to \n so that we can work on them separately. 128 | sed 's_]*>__g' | sed 's___g' | sed '/_\n_g' | 129 | # Prepend the series lines with \x01 to mark them and move them to last at the end. And remove the tags around them. 130 | sed '/]*&column=series/s_^_\x01_' | sed '/]*&column=series/s_]*>__g' | 131 | # Prepend the book lines with \x02 to mark them. And remove the tags around them. 132 | sed '/]*>__g' | 133 | # Prepend the ISBNs with \x03 to mark them and move them to last at the end. Also remove the tags which are at the end. 134 | # Its the only one that starts with a space. 135 | sed '/^\s/s_^\s\(.*\)_\x03\1_' | 136 | # -- Extracting links -- 137 | # Now remaining tags are one with the links. Extract links and remove [edit] link which is meant for Libgen Librarian. 138 | sed 's_]*href='\''\([^'\'']*\)'\''[^>]*[^<]*_\1 _g' | sed '/https:\/\/library.bz\/main\/edit.*/d' | 139 | # -- Cleanup -- 140 | # Convert & to ',', convert
tags and   to spaces. 141 | sed 's_&_,_g;s_
_ _g;s_ _ _g' | 142 | # Convert all newlines to tabspaces and all opening tags to newlines to separate books. Also remove trailing whitespaces. 143 | tr '\n' '\t' | sed 's_]*>_\n_g' | sed 's_\s*$__' | 144 | # -- Moving Series and ISBNs to last -- 145 | sed 's_\(\t\x01[^\t]*\)\?\(\t\x02[^\t]*\)\(\t\x03[^\t]*\)\?\(.*\)_\2\4\t\1\3_' | 146 | sed 's_\t\x01_\x01_' >>"$data_file" 147 | printf "\n" >>"$data_file" 148 | printf "\033[32mCompleted!\033[0m\n" 149 | page=$((page + 1)) 150 | done 151 | 152 | # -- DISPLAYING IN FZF -- 153 | # ID, Author, Title, Publisher, Year, Pages, Language, Size, Extension, Links, [Series], [ISBNs] -> 1,2,3,4,5,6,7,8,9,10,11,12 154 | id=$(awk -F "\t" '{printf("%-30.30s | %-70.70s | %-16.16s | %-7.7s | %-4.4s | %-s\n", $2, $3, $5, $8, $9, $1)}' "$data_file" | 155 | fzf --reverse --ansi --preview 'awk -F"\t" '\''$1 == '\''{-1}'\'' \ 156 | {printf("\033[1m\033[31mID:\033[0m %s\ 157 | \n\033[1m\033[31mAuthors:\033[0m %s\ 158 | \n\033[1m\033[31mTitle:\033[0m %s\ 159 | \n\t\033[1m\033[35mSeries:\033[0m %s\ 160 | \n\t\033[1m\033[35mISBNs:\033[0m %s\ 161 | \n\033[1m\033[31mPublisher:\033[0m %s\ 162 | \n\033[1m\033[31mYear:\033[0m %s\ 163 | \n\033[1m\033[31mPages:\033[0m %s\ 164 | \n\033[1m\033[31mLanguage:\033[0m %s\ 165 | \n\033[1m\033[31mSize:\033[0m %s\ 166 | \n\033[1m\033[31mExtension:\033[0m %s", $1,$2,$3,$11,$12,$4,$5,$6,$7,$8,$9)}'\'' '"$data_file"'' \ 167 | --preview-window=up:35% | 168 | awk '{print $NF}') 169 | [ -z "$id" ] && rm -f "$original_page" "$data_file" && exit 0 170 | 171 | # -- DOWNLOADING -- 172 | download_page="$(mktemp)" 173 | dl_links=$(awk -F"\t" '$1=="'"$id"'" {print $10}' "$data_file") 174 | final_link=$(printf "$dl_links" | tr ' ' '\n' | awk '{printf "[%d]\t%s\n", NR, $0}' | fzf --reverse | awk '{print $NF}') 175 | [ -z "$final_link" ] && rm -f "$original_page" "$data_file" "$download_page" && exit 0 176 | curl -s -L "$final_link" >"$download_page" 177 | 178 | # Get the main download link [GET] 179 | GET_URL=$(sed -n '/GET/s_.*.*GET.*.*_\1_p' "$download_page") 180 | 181 | # Give option to select ipfs gateways (Cloudflare, IPFS.io, Crust, Pinata) if first link selected 182 | if printf "%s" "$final_link" | grep -q 'books.ms'; then 183 | ipfs_gateways=$(sed -n '/Cloudflare/p' "$download_page" | sed 's__\n_g' | sed 's_.*\(.*\).*_\2\t\1_;$d') 184 | DOWNLOAD_URL=$(printf "[GET]\t%s\n%s" "$GET_URL" "$ipfs_gateways" | 185 | awk -F"\t" '{printf("%-20.20s %s\n", $1, $2)}' | fzf --reverse | awk '{print $NF}') 186 | [ -z "$DOWNLOAD_URL" ] && rm -f "$original_page" "$data_file" "$download_page" && exit 0 187 | wget --content-disposition "$DOWNLOAD_URL" 188 | else 189 | wget --content-disposition "https://libgen.gs/$GET_URL" 190 | fi 191 | 192 | rm -f "$original_page" "$data_file" "$download_page" 193 | 194 | printf "\nHere is the final link in case it didn't work:\n%s\n" "$final_link" 195 | --------------------------------------------------------------------------------