├── curlfire ├── LICENSE ├── SNIPPET-LICENSE ├── cookiefire └── README.md /curlfire: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -o errexit 4 | set -o nounset 5 | set -o pipefail 6 | 7 | profile=default 8 | skip= 9 | args=() 10 | for var in "$@"; do 11 | # Ignore known bad arguments 12 | case "$var" in 13 | -P) 14 | skip=yes 15 | ;; 16 | *) 17 | if [ -z "$skip" ]; then 18 | args+=("$var") 19 | else 20 | profile=$var 21 | fi; 22 | skip= 23 | esac; 24 | done 25 | 26 | 27 | curlcookies="$(mktemp /tmp/curlcookies.XXXXXXXXXX)" 28 | cookiefire "$profile" > "$curlcookies" 29 | curl -b "$curlcookies" "${args[@]}" ; 30 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2017 Tal Wrii 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 4 | 5 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 6 | 7 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 8 | -------------------------------------------------------------------------------- /SNIPPET-LICENSE: -------------------------------------------------------------------------------- 1 | This copyright notice applies only to the code snippets contained here https://superuser.com/a/1239036/653515, adapted for use in this project 2 | 3 | Copyright 2017 Hackerb9 (https://superuser.com/users/400780/hackerb9) 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 6 | 7 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 10 | -------------------------------------------------------------------------------- /cookiefire: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # -*- mode:sh -*- 3 | 4 | die() { 5 | echo >&2 "$*" 6 | exit 1 7 | } 8 | 9 | 10 | 11 | cleanup() { 12 | rm -f "$tmpfile" 13 | } 14 | trap cleanup EXIT INT QUIT TERM 15 | 16 | # Run older ld (pseudo condition) 17 | 18 | if [ "$#" == "0" ]; then 19 | profile=default 20 | elif [ "$#" == "1" ]; then 21 | profile=$1 22 | else 23 | die "usage $0 [profile]" 24 | fi; 25 | 26 | 27 | extract_cookies() { 28 | 29 | if [ "$#" -ge 1 ]; then 30 | sqlfile="$1" 31 | else 32 | if tty -s; then 33 | sqlfile=$(ls -t ~/.mozilla/firefox/*/cookies.sqlite | head -1) 34 | 35 | sqlfile="-" # Will use 'cat' below to read stdin 36 | fi 37 | fi 38 | 39 | if [ "$sqlfile" != "-" -a ! -r "$sqlfile" ]; then 40 | echo "Error. File $sqlfile is not readable." >&2 41 | exit 1 42 | fi 43 | 44 | # We have to copy cookies.sqlite, because FireFox has a lock on it 45 | cat "$sqlfile" >> $tmpfile 46 | 47 | 48 | # This is the format of the sqlite database: 49 | # CREATE TABLE moz_cookies (id INTEGER PRIMARY KEY, name TEXT, value TEXT, host TEXT, path TEXT,expiry INTEGER, lastAccessed INTEGER, isSecure INTEGER, isHttpOnly INTEGER); 50 | 51 | echo "# Netscape HTTP Cookie File" 52 | sqlite3 -separator $'\t' $tmpfile <<- EOF 53 | .mode tabs 54 | .header off 55 | select host, 56 | case substr(host,1,1)='.' when 0 then 'FALSE' else 'TRUE' end, 57 | path, 58 | case isSecure when 0 then 'FALSE' else 'TRUE' end, 59 | expiry, 60 | name, 61 | value 62 | from moz_cookies; 63 | EOF 64 | 65 | cleanup 66 | 67 | } 68 | 69 | tmpfile="$(mktemp /tmp/cookies.sqlite.XXXXXXXXXX)" 70 | curlcookies="$(mktemp /tmp/curlcookies.XXXXXXXXXX)" 71 | echo $HOME/.mozilla/firefox/*.$profile/cookies.sqlite | { read cookie_file ; 72 | extract_cookies "$cookie_file" ; 73 | } 74 | 75 | 76 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # curlfire 2 | @readwithai - [X](https://x.com/readwithai) - [blog](https://readwithai.substack.com) 3 | 4 | Run curl with the current [Firefox](https://www.mozilla.org/en-US/firefox/) cookies. 5 | This is useful for interacting with logged in websites from the shell, without having to manually deal with the login process. 6 | 7 | The executable `cookiefire` included in this package can be used to extract the cookies from Firefox. 8 | 9 | This is achieved by reading the **cookies.sqlite** file in Firefox profiles. 10 | 11 | # Attribution 12 | This code is adapted from [this Stack Exchange answer](https://superuser.com/a/1239036/653515) by [hackerb9](https://superuser.com/users/400780/hackerb9). 13 | 14 | # Usage 15 | ``` 16 | # Fetch google with the cookies from the default profile 17 | curlfire http://www.google.com/ 18 | 19 | # Fetch google with the cookies from the blah profile 20 | curlfire -P blah http://www.google.com/ 21 | 22 | # Getting cookies 23 | cookiefire > /tmp/cookies 24 | curl -b /tmp/cookies http://www.google.com/ 25 | ``` 26 | 27 | # Caveats 28 | * **Does not work with session cookies**[(1)](https://support.mozilla.org/en-US/questions/899388 29 | ) (you may be able to work around this by setting "Remember me" for the website with which you are using your tool) 30 | * Only tested on linux machines 31 | * Unlikely to work with windows 32 | * Will probably work on macs but untested (feedback welcome) 33 | 34 | # Installation 35 | ``` 36 | cd ~ 37 | git clone https://github.com/talwrii/curlfire 38 | echo 'PATH=$PATH:~/curlfire' >> ~/.bashrc 39 | ``` 40 | 41 | # Alternatives and prior work 42 | * Adapted from [this Stack Exchange answer](https://superuser.com/questions/666167/how-do-i-use-firefox-cookies-with-wget) 43 | * Firebug and friends allow one to copy requests as curl commands. This can be suitable for debugging. 44 | * Cookies can be exported manually from within Firefox 45 | 46 | All of these approaches can be time consuming when automating tasks. 47 | 48 | [Prior to Firefox 57](https://support.mozilla.org/en-US/kb/frequently-asked-questions-firefox-addon), the [remote control extension](https://addons.mozilla.org/en-US/firefox/addon/remote-control/) could be used to interact with the page currently viewed in Firefox from the shell (including outputting the current [document object model](https://en.wikipedia.org/wiki/Document_Object_Model) as HTML). 49 | 50 | # Complementary projects (and shameless advertising) 51 | Users of this project might also be interested in: 52 | 53 | * [clixpath](https://github.com/talwrii/clixpath) (by author) - an open-source tool for extracting information out of HTML documents in machine-readable JSON using [Xpath](https://www.w3.org/TR/1999/REC-xpath-19991116/). 54 | * [PhantomJs](http://phantomjs.org/) - a scriptable command-line only browser than can be used to [render the document object model in Javascript based-websites](https://stackoverflow.com/a/9978162) and output more easily processed HTML. 55 | 56 | # Copyright 57 | This code is distributed under [an MIT license](LICENSE). 58 | It contains code adapted from a code snippet in the aforementioned Stack Exchange answer which is also distributed under [an MIT license](SNIPPET-LICENSE) as [stipulated by the Stack Exchange terms and conditions](https://meta.stackexchange.com/questions/272956/a-new-code-license-the-mit-this-time-with-attribution-required). 59 | 60 | # About me 61 | I am [@readwithai](https://x.com/readwithai). I make tools for [productivity](https://readwithai.substack.com/p/obsidian-plugin-repl) and agency particularly related to deep reading and using [Obsidian](https://readwithai.substack.com/p/what-exactly-is-obsidian). If this sounds interesting you can follow me on [X](https://x.com/readwithai) or [substack](https://readwithai.substack.com). 62 | 63 | If you find *this* piece of software useful you could consider paying me 2 dollars on my [ko-fi](https://ko-fi.com/readwithai). 64 | 65 | 66 | --------------------------------------------------------------------------------