├── .travis.yml ├── CHANGELOG.md ├── LICENSE ├── Makefile ├── README.md ├── hist └── test ├── Makefile ├── _init.sh ├── run.sh ├── test_exit_codes.sh ├── test_import.sh └── test_search.sh /.travis.yml: -------------------------------------------------------------------------------- 1 | script: make test 2 | language: c 3 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # historian changelog 2 | 3 | ## 0.0.2 4 | 5 | * Renames global variables to `HISTORIAN_SRC`, `HISTORIAN_DB`, 6 | `HISTORIAN_SQLITE3` and makes them overridable from outside the 7 | script 8 | 9 | * Fixes import delimiter bug to allow for backtick (but disallow 0x01) 10 | 11 | ## 0.0.1 12 | 13 | * Initial implementation 14 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2017 Jerry Chen. All rights reserved. 2 | 3 | Redistribution and use in source and binary forms, with or without 4 | modification, are permitted provided that the following conditions are 5 | met: 6 | 7 | 1. Redistributions of source code must retain the above copyright 8 | notice, this list of conditions and the following disclaimer. 9 | 10 | 2. Redistributions in binary form must reproduce the above 11 | copyright notice, this list of conditions and the following 12 | disclaimer in the documentation and/or other materials provided 13 | with the distribution. 14 | 15 | THIS SOFTWARE IS PROVIDED BY JERRY CHEN ``AS IS'' AND ANY EXPRESS OR 16 | IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18 | DISCLAIMED. IN NO EVENT SHALL JERRY CHEN OR CONTRIBUTORS BE LIABLE FOR 19 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE 21 | GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 22 | INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 23 | IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR 24 | OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF 25 | ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | 27 | The views and conclusions contained in the software and documentation 28 | are those of the authors and should not be interpreted as representing 29 | official policies, either expressed or implied, of Jerry Chen. 30 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: test 2 | all: 3 | test: 4 | @cd test && make --no-print-directory env test 5 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | historian 2 | ========= 3 | 4 | Command-line utility for managing shell history in a SQLite database. 5 | 6 | `~/.bash_history` is deduped and imported into a database. 7 | 8 | ### Requirements 9 | 10 | * SQLite 11 | * a home directory 12 | 13 | ### Installation 14 | 15 | Download and 1) add the directory to your `$PATH` 16 | 17 | export PATH="$PATH:/Users/jerry/historian" 18 | 19 | or 2) add `hist` as an alias. 20 | 21 | alias hist="/Users/jerry/historian/hist" 22 | 23 | ### Getting Started 24 | 25 | Import your `~/.bash_history` 26 | 27 | $ hist import 28 | 29 | ### Super Installation 30 | 31 | Add `hist import` to your `.profile` (assuming `hist` is in your path): 32 | 33 | $ echo hist import >> ~/.profile 34 | 35 | This will import your .bash_history every time you launch a new shell. 36 | 37 | ### Usage 38 | 39 | Show config: 40 | 41 | $ hist config 42 | version: 0.0.2 43 | bash_history: /Users/jerry/.bash_history 44 | db: /Users/jerry/.historian.db 45 | sqlite3: /opt/local/bin/sqlite3 46 | 47 | Search: 48 | 49 | $ hist search monsters 50 | 690 51 | echo a zombie with no conscience >> ~/monsters 52 | 689 53 | echo ghoul >> ~/monsters 54 | 688 55 | echo goblin >> ~/monsters 56 | 687 57 | echo lochness >> ~/monsters 58 | 59 | Search (shorthand): 60 | 61 | $ hist /monsters 62 | 690 63 | echo a zombie with no conscience >> ~/monsters 64 | 689 65 | echo ghoul >> ~/monsters 66 | 688 67 | echo goblin >> ~/monsters 68 | 687 69 | echo lochness >> ~/monsters 70 | 71 | View log: 72 | 73 | $ hist log 74 | 1020 75 | rm -f README.md 76 | 1019 77 | emacs README.md 78 | 1018 79 | rm -rf .git 80 | 81 | ### Pitfalls 82 | 83 | Live like your db file could be corrupted at any time. 84 | 85 | Be wary of running specially crafted `hist` commands or against 86 | `~/.bash_history` files. 87 | 88 | ### Cool Things in the Future 89 | 90 | * `export` to append to `~/.bash_history` 91 | * `scrub` items from history 92 | * set or autodetect configs 93 | * other shells than bash 94 | * timestamp support 95 | -------------------------------------------------------------------------------- /hist: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | VERSION="0.0.2" 4 | HISTORIAN_SRC=${HISTORIAN_SRC-"${HOME}/.bash_history"} 5 | HISTORIAN_DB=${HISTORIAN_DB-"${HOME}/.historian.db"} 6 | HISTORIAN_SQLITE3=${HISTORIAN_SQLITE3-"$(which sqlite3)"} 7 | MAGIC="$(echo -e "\x10\x83\xB9\x9F\x34\xB5\x96\x45")" # 0118 999 881 999 119 725 3 8 | MAGIC_ENUM_QUOTE=1 9 | 10 | SEPARATOR=$(echo -e "\x01") 11 | 12 | # Other ENV parameters: 13 | # 14 | # - ZSH_EXTENDED_HISTORY: if set, parses HISTORIAN_SRC using zsh's 15 | # EXTENDED_HISTORY format 16 | 17 | usage() { 18 | echo "Usage: hist " >&2 19 | echo "subcommands:" >&2 20 | echo " config show config" >&2 21 | echo " count count items in history" >&2 22 | echo " import import to db" >&2 23 | echo " shell launch sqlite3 shell with db" >&2 24 | echo " search search for " >&2 25 | echo " /term search for " >&2 26 | echo " version show the version" >&2 27 | } 28 | 29 | preflight_check() { 30 | if [ -z "$HOME" ]; then 31 | echo "need \$HOME" >&2 32 | exit 1 33 | fi 34 | 35 | if [ -z "${HISTORIAN_SQLITE3}" ]; then 36 | echo "need sqlite3" >&2 37 | exit 1 38 | fi 39 | } 40 | 41 | ensure_db_exists() { 42 | ( cat < "${sanitized_src}" \ 97 | ; 98 | if [ -n "${ZSH_EXTENDED_HISTORY}" ]; then 99 | _import_zsh_extended_history; 100 | else 101 | _import_default; 102 | fi 103 | } 104 | 105 | _import_default() { 106 | ( cat < ${tmp_src} 132 | ( cat <&2 197 | "${HISTORIAN_SQLITE3}" "${HISTORIAN_DB}"; 198 | } 199 | 200 | cmd_version() { 201 | echo "historian version: ${VERSION}" 202 | } 203 | 204 | main() { 205 | local cmd=$1 206 | shift 207 | case $cmd in 208 | config) 209 | cmd_config $@ 210 | ;; 211 | count) 212 | cmd_count $@ 213 | ;; 214 | import) 215 | cmd_import $@ 216 | ;; 217 | log) 218 | cmd_log $@ 219 | ;; 220 | search) 221 | cmd_search $@ 222 | ;; 223 | shell) 224 | cmd_shell $@ 225 | ;; 226 | version) 227 | cmd_version $@ 228 | ;; 229 | "") 230 | usage 231 | ;; 232 | *) 233 | if [ -n "$(echo "$cmd" | grep -E '^/')" ]; then 234 | cmd_search_slash $cmd $@ 235 | else 236 | usage 237 | exit 1 238 | fi 239 | ;; 240 | esac 241 | } 242 | 243 | main $@ 244 | -------------------------------------------------------------------------------- /test/Makefile: -------------------------------------------------------------------------------- 1 | all: env test 2 | env: 3 | uname -a 4 | @echo 5 | sqlite3 --version 6 | @echo 7 | awk --version 8 | @echo 9 | test: 10 | @./run.sh 11 | -------------------------------------------------------------------------------- /test/_init.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | HIST=$(dirname $0)/../hist 4 | 5 | fail=false 6 | success_count=0 7 | failure_count=0 8 | 9 | begin_tests() { 10 | reset_counts 11 | reset_workspace 12 | } 13 | 14 | reset_counts() { 15 | fail=false 16 | success_count=0 17 | failure_count=0 18 | } 19 | 20 | reset_workspace() { 21 | WORKSPACE=$(mktemp -d) 22 | } 23 | 24 | new_sandbox() { 25 | mktemp -d 26 | } 27 | 28 | sandbox_sql() { 29 | if [ -z "$sandbox" ]; then 30 | "\$sandbox required. Aborting" 31 | exit 1 32 | fi 33 | 34 | echo "$@" | sqlite3 $sandbox/.historian.db 35 | } 36 | 37 | sandbox_hist() { 38 | if [ -z "$sandbox" ]; then 39 | "\$sandbox required. Aborting" 40 | exit 1 41 | fi 42 | HOME=$sandbox \ 43 | $HIST $@ >/dev/null 2>&1 44 | rv=$? 45 | return $? 46 | } 47 | 48 | sandbox_hist_with_output() { 49 | if [ -z "$sandbox" ]; then 50 | "\$sandbox required. Aborting" 51 | exit 1 52 | fi 53 | HOME=$sandbox \ 54 | $HIST $@ 55 | rv=$? 56 | return $? 57 | } 58 | 59 | add_success() { 60 | let success_count=success_count+1 61 | } 62 | 63 | add_failure() { 64 | let failure_count=failure_count+1 65 | local msg="$1" 66 | if [ -n "${CURRENT_TEST_FN}" ]; then 67 | echo "${CURRENT_TEST_FN} : Failed: ${msg}" 68 | else 69 | echo "Failed: ${msg}" 70 | fi 71 | } 72 | 73 | destroy_sandbox() { 74 | local sandbox=$1 75 | if [ -n $sandbox ] && [ -n "$(echo $sandbox | grep /tmp)" ]; then 76 | rm -rf $sandbox 77 | else 78 | echo "Warning: not destroying sandbox: ${sandbox}" >&2 79 | fi 80 | } 81 | 82 | assert_equal() { 83 | local expected="$1" 84 | local actual="$2" 85 | local msg="$3" 86 | 87 | if [ "$expected" != "$actual" ]; then 88 | add_failure "expected ${expected}, but got ${actual}: ${msg}" 89 | else 90 | add_success 91 | fi 92 | } 93 | 94 | finish_tests() { 95 | let total_count=${failure_count}+${success_count} 96 | 97 | echo "${success_count} / ${total_count} assertions passed" 98 | 99 | if [ ${failure_count} -gt 0 ]; then 100 | exit 1 101 | fi 102 | } 103 | -------------------------------------------------------------------------------- /test/run.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | source $(dirname $0)/_init.sh 4 | 5 | begin_tests 6 | 7 | run_tests() { 8 | local test_dir=$1 9 | local test_fn_prefix=$2 10 | 11 | for file in ${test_dir}/test_*.sh; do 12 | if [ "$(basename $file)" == "$(basename $0)" ]; then 13 | # don't run myself 14 | continue 15 | fi 16 | 17 | echo "$(basename $file):" 18 | 19 | before_test_fns=$(declare -F | awk '{print $NF}' | grep -E ^${test_fn_prefix}_) 20 | source $file 21 | after_test_fns=$(declare -F | awk '{print $NF}' | grep -E ^${test_fn_prefix}_) 22 | 23 | test_fns=$(comm -1 <(echo "${before_test_fns}") <(echo "${after_test_fns}")) 24 | max_test_fn_size=0 25 | for test_fn in $test_fns; do 26 | len=$(echo $test_fn | awk '{print length($0)}') 27 | if [ $len -gt $max_test_fn_size ]; then 28 | max_test_fn_size=$len 29 | fi 30 | done 31 | for test_fn in $test_fns; do 32 | echo -n " ${test_fn} ... " 33 | sandbox=$(new_sandbox) 34 | last_failure_count=${failure_count} 35 | CURRENT_TEST_FN=$test_fn 36 | eval $test_fn 37 | let new_failure_count=${failure_count}-${last_failure_count} 38 | if [ ${new_failure_count} -gt 0 ]; then 39 | echo "FAIL" 40 | else 41 | echo "PASS" 42 | fi 43 | destroy_sandbox $sandbox 44 | done 45 | done 46 | } 47 | 48 | run_tests $(dirname $0) htest 49 | 50 | finish_tests 51 | -------------------------------------------------------------------------------- /test/test_exit_codes.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | htest_bare_command_returns_zero_exit_code() { 4 | sandbox_hist 5 | assert_equal 0 $? "bare command should return exit code 0" 6 | } 7 | 8 | htest_version_returns_zero_exit_code() { 9 | sandbox_hist version 10 | assert_equal 0 $? "version should return exit code 0" 11 | } 12 | -------------------------------------------------------------------------------- /test/test_import.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | htest_import_simple() { 4 | cat >> $sandbox/.bash_history <> $sandbox/.bash_history <> $sandbox/.bash_history < $sandbox/exported_commands.txt 46 | diff $sandbox/.bash_history $sandbox/exported_commands.txt 47 | assert_equal 0 $? "exported commands should match" 48 | } 49 | 50 | htest_import_run_twice_will_do_nothing_the_second_time() { 51 | cat >> $sandbox/.bash_history <> $sandbox/.bash_history < $tmp 81 | diff $tmp $sandbox/.bash_history 82 | assert_equal 0 $? 83 | rm -f $tmp 84 | } 85 | 86 | htest_import_zsh_extended_history_parses_correctly() { 87 | cat >> $sandbox/.bash_history <> $sandbox/expected_commands.psv < $sandbox/actual_commands.psv; 118 | diff $sandbox/expected_commands.psv $sandbox/actual_commands.psv; 119 | assert_equal 0 $? "rows imported with ZSH_EXTENDED_HISTORY set should match" 120 | } 121 | -------------------------------------------------------------------------------- /test/test_search.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | htest_search_length() { 4 | local length=1024 5 | echo \ 6 | | awk '{while (z++ < '$length') {printf "A"}}' \ 7 | > $sandbox/.bash_history 8 | sandbox_hist import 9 | actual=$(sandbox_hist_with_output search A \ 10 | | grep A \ 11 | | awk '{print $NF}' \ 12 | ); 13 | actual_length=$(echo $actual | awk '{print length($0);}') 14 | assert_equal ${actual_length} $length "actual length" 15 | } 16 | --------------------------------------------------------------------------------