├── README.md └── timeshift.py /README.md: -------------------------------------------------------------------------------- 1 | # timeshift 2 | A python script to shift the timestamp on syslog and httpd log data. Useful for forensicators combating time skew, time zones, and other such foolery. 3 | 4 | ## Usage 5 | 6 | ``` 7 | $ timeshift.py --help 8 | usage: timeshift.py [-h] [-m {syslog,httpdlog,rfc3339,cobaltstrike}] 9 | [-o OFFSET] [-i {second,minute,hour,day}] [-y YEAR] 10 | [-r INFILE] [-w OUTFILE] 11 | 12 | Shift the date for all entries in an input data set by a specified interval of 13 | time. Offset and interval options are required when using syslog mode. 14 | 15 | optional arguments: 16 | -h, --help show this help message and exit 17 | -m {syslog,httpdlog,rfc3339,cobaltstrike}, --mode {syslog,httpdlog,rfc3339,cobaltstrike} 18 | Type of timestamp to seek and adjust (default = 19 | syslog) 20 | -o OFFSET, --offset OFFSET 21 | Amount of time to shift (pos/neg integer, only 22 | required for "syslog" mode 23 | -i {second,minute,hour,day}, --interval {second,minute,hour,day} 24 | Interval of time to shift (only required for "syslog" 25 | and "cobaltstrike" modes 26 | -y YEAR, --year YEAR Year to assume (default 2018) 27 | -r INFILE, --infile INFILE 28 | Input file to process (default STDIN) 29 | -w OUTFILE, --outfile OUTFILE 30 | Output file to create - will be overwritten if exists 31 | (default STDOUT) 32 | ``` 33 | 34 | ### Example Usage 35 | Original contents of syslog file: 36 | 37 | ``` 38 | $ cat maillog 39 | Jun 8 15:20:02 proxy sendmail[2295]: alias database /etc/aliases rebuilt by root 40 | Jun 8 15:20:02 proxy sendmail[2295]: /etc/aliases: 76 aliases, longest 10 bytes, 765 bytes total 41 | Jun 8 15:20:02 proxy sendmail[2300]: starting daemon (8.13.8): SMTP+queueing@01:00:00 42 | Jun 8 15:20:02 proxy sm-msp-queue[2308]: starting daemon (8.13.8): queueing@01:00:00 43 | ``` 44 | 45 | Assuming source file is reflected in EDT (UTC-0400), change to UTC (as it should be!): 46 | ``` 47 | $ timeshift.py -m syslog -o 4 -i hour -r maillog 48 | Jun 8 19:20:02 proxy sendmail[2295]: alias database /etc/aliases rebuilt by root 49 | Jun 8 19:20:02 proxy sendmail[2295]: /etc/aliases: 76 aliases, longest 10 bytes, 765 bytes total 50 | Jun 8 19:20:02 proxy sendmail[2300]: starting daemon (8.13.8): SMTP+queueing@01:00:00 51 | Jun 8 19:20:02 proxy sm-msp-queue[2308]: starting daemon (8.13.8): queueing@01:00:00 52 | ``` 53 | 54 | Correct sendmail entries in source file to account for the system's clock being 23 seconds fast 55 | ``` 56 | $ grep sendmail maillog | ./timeshift.py -m syslog -o -23 -i second 57 | Jun 8 15:19:39 proxy sendmail[2295]: alias database /etc/aliases rebuilt by root 58 | Jun 8 15:19:39 proxy sendmail[2295]: /etc/aliases: 76 aliases, longest 10 bytes, 765 bytes total 59 | Jun 8 15:19:39 proxy sendmail[2300]: starting daemon (8.13.8): SMTP+queueing@01:00:00 60 | ``` 61 | 62 | Original contents of HTTPD access log file: 63 | ``` 64 | $ cat lewestech.com-access 65 | 82.220.38.35 - - [11/Oct/2015:10:42:02 +0400] "POST /wp-login.php HTTP/1.1" 200 4697 "http://lewestech.com/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" 66 | 208.115.113.85 - - [11/Oct/2015:11:27:15 +0400] "GET /clients/clients/waggies-by-maggie HTTP/1.1" 301 128 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)" 67 | 65.254.225.173 - - [11/Oct/2015:11:29:49 +0400] "POST /wp-login.php HTTP/1.1" 200 4697 "http://lewestech.com/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" 68 | 82.239.166.225 - - [11/Oct/2015:11:58:49 +0400] "GET /tag/for572/ HTTP/1.1" 200 24951 "https://www.google.fr" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0" 69 | 82.239.166.225 - - [11/Oct/2015:11:58:50 +0400] "GET /wp-content/themes/lewestech/style.css HTTP/1.1" 200 10512 "http://lewestech.com/tag/for572/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0" 70 | 82.239.166.225 - - [11/Oct/2015:11:58:51 +0400] "GET /wp-content/themes/lewestech/scripts/utils.js HTTP/1.1" 200 123 "http://lewestech.com/tag/for572/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0" 71 | ``` 72 | 73 | Convert HTTPD access log with UTC offset to UTC native 74 | ``` 75 | $ cat lewestech.com-access | ./timeshift.py -m httpd 76 | 82.220.38.35 - - [11/Oct/2015:06:42:02 +0000] "POST /wp-login.php HTTP/1.1" 200 4697 "http://lewestech.com/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" 77 | 208.115.113.85 - - [11/Oct/2015:07:27:15 +0000] "GET /clients/clients/waggies-by-maggie HTTP/1.1" 301 128 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)" 78 | 65.254.225.173 - - [11/Oct/2015:07:29:49 +0000] "POST /wp-login.php HTTP/1.1" 200 4697 "http://lewestech.com/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" 79 | 82.239.166.225 - - [11/Oct/2015:07:58:49 +0000] "GET /tag/for572/ HTTP/1.1" 200 24951 "https://www.google.fr" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0" 80 | 82.239.166.225 - - [11/Oct/2015:07:58:50 +0000] "GET /wp-content/themes/lewestech/style.css HTTP/1.1" 200 10512 "http://lewestech.com/tag/for572/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0" 81 | 82.239.166.225 - - [11/Oct/2015:07:58:51 +0000] "GET /wp-content/themes/lewestech/scripts/utils.js HTTP/1.1" 200 123 "http://lewestech.com/tag/for572/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0" 82 | ``` 83 | 84 | Original contents of RFC3339 timestamp file: 85 | ``` 86 | $ cat messages 87 | <5>2016-05-05T23:12:09.649085-05:00 quaff kernel:[27198521.247185] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21110 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 88 | <5>2016-05-05T23:12:09.649157-05:00 quaff kernel:[27198521.247213] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21111 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 89 | <5>2016-05-05T23:12:09.649161-05:00 quaff kernel:[27198521.247228] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21112 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 90 | <5>2016-05-05T23:12:09.649163-05:00 quaff kernel:[27198521.247252] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21113 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 91 | <5>2016-05-05T23:12:09.649165-05:00 quaff kernel:[27198521.247273] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21114 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 92 | ``` 93 | 94 | Convert RFC3339 timestamps with UTC offset to UTC native 95 | ``` 96 | $ ./timeshift.py -m rfc3339 -r messages 97 | <5>2016-05-06T04:12:09.649085+00:00 quaff kernel:[27198521.247185] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21110 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 98 | <5>2016-05-06T04:12:09.649157+00:00 quaff kernel:[27198521.247213] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21111 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 99 | <5>2016-05-06T04:12:09.649161+00:00 quaff kernel:[27198521.247228] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21112 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 100 | <5>2016-05-06T04:12:09.649163+00:00 quaff kernel:[27198521.247252] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21113 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 101 | <5>2016-05-06T04:12:09.649165+00:00 quaff kernel:[27198521.247273] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21114 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 102 | ``` 103 | 104 | Original contenst of Cobalt Strike log file: 105 | ``` 106 | $ cat cobaltstrike.txt 107 | 08/23 21:38:35 [input] download file.zip 108 | 08/23 21:38:35 [task] Tasked beacon to download file.zip 109 | 08/23 21:38:42 [checkin] host called home, sent: 37 bytes 110 | 08/23 21:38:42 [output] 111 | started download of C:\Users\victim\Documents\file.zip (14892 bytes) 112 | 113 | 08/23 21:38:42 [output] 114 | download of file.zip is complete 115 | 116 | 08/23 21:39:40 [input] ls 20180823 117 | 08/23 21:39:40 [task] Tasked beacon to list files in 20180823 118 | 08/23 21:39:43 [checkin] host called home, sent: 26 bytes 119 | ``` 120 | 121 | Assuming source file is reflected in EDT (UTC-0400), change to UTC (as it should be!): 122 | ``` 123 | $ timeshift.py -m cobaltstrike -o 4 -i hour -r cobaltstrike.txt 124 | 2018-08-24T01:38:35 [input] download file.zip 125 | 2018-08-24T01:38:35 [task] Tasked beacon to download file.zip 126 | 2018-08-24T01:38:42 [checkin] host called home, sent: 37 bytes 127 | 2018-08-24T01:38:42 [output] 128 | started download of C:\Users\victim\Documents\file.zip (14892 bytes) 129 | 130 | 2018-08-24T01:38:42 [output] 131 | download of file.zip is complete 132 | 133 | 2018-08-24T01:39:40 [input] ls 20180823 134 | 2018-08-24T01:39:40 [task] Tasked beacon to list files in 20180823 135 | 2018-08-24T01:39:43 [checkin] host called home, sent: 26 bytes 136 | ``` -------------------------------------------------------------------------------- /timeshift.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # (C)2018 Lewes Technology Consulting, LLC 3 | import sys 4 | from datetime import datetime, timedelta, date 5 | import argparse 6 | import re 7 | 8 | intervals = ('second', 'minute', 'hour', 'day') 9 | conversion_modes = ('syslog', 'httpdlog', 'rfc3339', 'cobaltstrike') 10 | curryear = date.today().year 11 | 12 | syslog_re = re.compile('(?P(?P[A-Za-z]{3} [0-9 ]{2}) (?P