├── test ├── dryrun_err.exp ├── replay_out.exp ├── replayfile ├── dryrun_out.exp ├── parse_csv.exp ├── parse_csv_out.exp ├── parse_err_out.exp ├── parse_err.exp ├── runtest.sh.in └── replay_err.exp ├── TODO ├── .gitignore ├── Dockerfile ├── LICENSE ├── Formula └── pgreplay.rb ├── windows.c ├── Makefile.in ├── configure.in ├── CHANGELOG ├── config.h.in ├── pgreplay.h ├── pgreplay.1 ├── replayitem.c ├── replayfile.c ├── main.c ├── README.md ├── pgreplay.html ├── install-sh ├── database.c └── config.sub /test/dryrun_err.exp: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/replay_out.exp: -------------------------------------------------------------------------------- 1 | Execution is 10 seconds behind schedule 2 | 3 | -------------------------------------------------------------------------------- /test/replayfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/laurenz/pgreplay/HEAD/test/replayfile -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | This is a list of things that could be improved. 2 | Help is welcome! 3 | 4 | - Use "single-row mode" with PQsetSingleRowMode. 5 | This would reduce the memory requirements for large result sets. 6 | 7 | Noted by Thomas Dziedzic in #15. 8 | -------------------------------------------------------------------------------- /test/dryrun_out.exp: -------------------------------------------------------------------------------- 1 | 2 | Replay statistics (dry run) 3 | =========================== 4 | 5 | Duration of recorded workload: 37.764 seconds 6 | Calls to the server: 37 7 | Total number of connections: 4 8 | Maximum number of concurrent connections: 4 9 | SQL statements executed: 18 10 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Object files 2 | *.o 3 | *.obj 4 | 5 | # Libraries 6 | *.lib 7 | *.a 8 | 9 | # Shared objects (inc. Windows DLLs) 10 | *.dll 11 | *.so 12 | *.so.* 13 | *.dylib 14 | 15 | # build products 16 | pgreplay 17 | pgreplay.exe 18 | Makefile 19 | config.h 20 | config.log 21 | config.status 22 | 23 | # test subdirectory 24 | test/ 25 | 26 | # other versioning systems 27 | .svn 28 | CVS 29 | 30 | # M4 cache 31 | autom4te.cache/ 32 | -------------------------------------------------------------------------------- /test/parse_csv.exp: -------------------------------------------------------------------------------- 1 | Warning: "SET client_encoding" statement ignored in line 9 2 | Warning: "SET client_encoding" statement ignored in line 11 3 | Warning: "SET client_encoding" statement ignored in line 24 4 | Warning: COPY statement ignored in line 28 5 | Warning: fast-path function call ignored in line 43 6 | Warning: fast-path function call ignored in line 44 7 | Warning: fast-path function call ignored in line 45 8 | Warning: fast-path function call ignored in line 46 9 | Warning: fast-path function call ignored in line 48 10 | -------------------------------------------------------------------------------- /test/parse_csv_out.exp: -------------------------------------------------------------------------------- 1 | 2 | Parse statistics 3 | ================ 4 | 5 | Log lines read: 58 6 | Total SQL statements processed: 25 7 | Simple SQL statements processed: 19 8 | (includes 1 ignored copy statements) 9 | Parametrized SQL statements processed: 4 10 | Named prepared SQL statements executions processed: 2 11 | Different named prepared SQL statements processed: 1 12 | (average reuse count 1.000) 13 | Cancel requests processed: 1 14 | Fast-path function calls ignored: 5 15 | Duration of recorded workload: 37.764 seconds 16 | -------------------------------------------------------------------------------- /test/parse_err_out.exp: -------------------------------------------------------------------------------- 1 | 2 | Parse statistics 3 | ================ 4 | 5 | Log lines read: 188 6 | Total SQL statements processed: 25 7 | Simple SQL statements processed: 19 8 | (includes 1 ignored copy statements) 9 | Parametrized SQL statements processed: 4 10 | Named prepared SQL statements executions processed: 2 11 | Different named prepared SQL statements processed: 1 12 | (average reuse count 1.000) 13 | Cancel requests processed: 1 14 | Fast-path function calls ignored: 5 15 | Duration of recorded workload: 37.764 seconds 16 | -------------------------------------------------------------------------------- /test/parse_err.exp: -------------------------------------------------------------------------------- 1 | Warning: "SET client_encoding" statement ignored in line 10 2 | Warning: "SET client_encoding" statement ignored in line 17 3 | Warning: "SET client_encoding" statement ignored in line 26 4 | Warning: COPY statement ignored in line 31 5 | Warning: fast-path function call ignored in line 50 6 | Warning: fast-path function call ignored in line 51 7 | Warning: fast-path function call ignored in line 52 8 | Warning: fast-path function call ignored in line 53 9 | Warning: fast-path function call ignored in line 55 10 | Found memory dump in line 60 11 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:22.04 2 | 3 | RUN TZ=UTC 4 | RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone 5 | RUN apt-get update && \ 6 | apt-get install --no-install-recommends -y tzdata make gcc libc6-dev postgresql-14 libpq-dev postgresql-doc-14 git ca-certificates && \ 7 | apt-get clean && rm -rf /var/lib/apt/lists/* 8 | 9 | WORKDIR /root 10 | RUN git clone https://github.com/laurenz/pgreplay.git 11 | WORKDIR /root/pgreplay 12 | RUN ./configure --with-postgres=/usr/bin 13 | RUN make 14 | RUN make install 15 | RUN ln -s /root/pgreplay/pgreplay /usr/local/bin 16 | 17 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Portions Copyright (c) 2017-2023, CYBERTEC PostgreSQL International GmbH 2 | Portions Copyright (c) 2010-2017, Magistrat der Stadt Wien 3 | 4 | Permission to use, copy, modify, and distribute this software and its 5 | documentation for any purpose, without fee, and without a written agreement 6 | is hereby granted, provided that the above copyright notice and this paragraph 7 | and the following two paragraphs appear in all copies. 8 | 9 | IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE TO ANY PARTY FOR 10 | DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING 11 | LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, 12 | EVEN IF THE COPYRIGHT HOLDER HAS BEEN ADVISED OF THE POSSIBILITY OF 13 | SUCH DAMAGE. 14 | 15 | THE COPYRIGHT HOLDER SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, 16 | BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 17 | A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, 18 | AND THE COPYRIGHT HOLDER HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, 19 | SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. 20 | -------------------------------------------------------------------------------- /Formula/pgreplay.rb: -------------------------------------------------------------------------------- 1 | class Pgreplay < Formula 2 | desc "Record and replay real-life PostgreSQL workloads" 3 | homepage "https://github.com/laurenz/pgreplay" 4 | url "https://github.com/laurenz/pgreplay/archive/refs/tags/PGREPLAY_1_3_0.tar.gz" 5 | sha256 "ff1d67d568df5a23c0ba79b74d7870f0f8711683cda8b9e0a1fafd18c2f33409" 6 | license "MIT" 7 | head "https://github.com/laurenz/pgreplay.git", branch: "master" 8 | 9 | livecheck do 10 | url :stable 11 | strategy :github_latest 12 | end 13 | 14 | depends_on "postgresql" 15 | 16 | def install 17 | pg_config = Formula["postgresql"].opt_bin/"pg_config" 18 | libdir = Utils.safe_popen_read(pg_config, "--libdir").strip 19 | includedir = Utils.safe_popen_read(pg_config, "--includedir").strip 20 | 21 | ENV.append "LDFLAGS", "-L#{libdir}" 22 | ENV.append "CPPFLAGS", "-I#{includedir}" 23 | 24 | system "./configure", "--prefix=#{prefix}", "--with-postgres=#{pg_config.dirname}" 25 | 26 | inreplace "Makefile" do |s| 27 | s.gsub! "-D -m 0755", "-m 0755" 28 | s.gsub! "-D -m 0644", "-m 0644" 29 | end 30 | 31 | bin.mkpath 32 | man1.mkpath 33 | 34 | system "make" 35 | system "make", "install" 36 | end 37 | 38 | test do 39 | assert_match version.to_s, shell_output("#{bin}/pgreplay -v") 40 | end 41 | end 42 | -------------------------------------------------------------------------------- /windows.c: -------------------------------------------------------------------------------- 1 | #include "pgreplay.h" 2 | #include 3 | #include 4 | 5 | /* gets the last error and prints an error message */ 6 | 7 | void win_perror(const char *prefix, int is_network_error) { 8 | DWORD error_nr; 9 | char *errmsg; 10 | /* catalog of Windows socket error messages */ 11 | static HMODULE sock_err_mod = NULL; 12 | 13 | /* get the message number */ 14 | if (is_network_error) { 15 | error_nr = WSAGetLastError(); 16 | 17 | if (NULL == sock_err_mod) { 18 | /* try to load the Windows socket error message catalog */ 19 | sock_err_mod = LoadLibraryEx( 20 | "netmsg.dll", 21 | NULL, 22 | LOAD_LIBRARY_AS_DATAFILE 23 | ); 24 | } 25 | } else { 26 | error_nr = GetLastError(); 27 | } 28 | 29 | /* get the error message text */ 30 | if (FormatMessage( 31 | FORMAT_MESSAGE_ALLOCATE_BUFFER 32 | | FORMAT_MESSAGE_IGNORE_INSERTS 33 | | FORMAT_MESSAGE_FROM_SYSTEM 34 | | ((is_network_error && sock_err_mod) ? FORMAT_MESSAGE_FROM_HMODULE : 0), 35 | sock_err_mod, 36 | error_nr, 37 | MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), 38 | (LPSTR) &errmsg, 39 | 0, 40 | NULL)) { 41 | fprintf(stderr, "%s: %s\n", prefix, errmsg); 42 | 43 | /* free the memory for the error message */ 44 | LocalFree(errmsg); 45 | } else { 46 | fprintf(stderr, "%s: error number %ld\n", prefix, error_nr); 47 | } 48 | } 49 | -------------------------------------------------------------------------------- /Makefile.in: -------------------------------------------------------------------------------- 1 | CFLAGS = @CFLAGS@ 2 | CPPFLAGS = @CPPFLAGS@ 3 | LDFLAGS = @LDFLAGS@ 4 | LIBS = @LIBS@ 5 | CC = @CC@ 6 | EXTRA_OBJS = @EXTRA_OBJS@ 7 | prefix = @prefix@ 8 | exec_prefix = @exec_prefix@ 9 | bindir = @bindir@ 10 | datarootdir = @datarootdir@ 11 | datadir = @datadir@ 12 | mandir = @mandir@ 13 | docdir = @docdir@ 14 | htmldir = @htmldir@ 15 | INSTALL = @INSTALL@ 16 | VERSION = @PACKAGE_VERSION@ 17 | EXE = pgreplay 18 | OBJS = parse.o replayitem.o main.o replayfile.o database.o $(EXTRA_OBJS) 19 | HEADERS = config.h pgreplay.h 20 | 21 | .PHONY: clean distclean tarball test install install_bin install_man install_html 22 | 23 | .SUFFIXES: .o .c .h 24 | 25 | $(EXE): $(OBJS) $(HEADERS) 26 | $(CC) -o $@ $(CFLAGS) $(CPPFLAGS) $(OBJS) $(LDFLAGS) $(LIBS) 27 | 28 | $(OBJS): $(HEADERS) 29 | 30 | clean: 31 | rm -f $(OBJS) $(EXE) 32 | 33 | distclean: clean 34 | rm -rf Makefile config.h config.h.in~ config.log config.cache config.status autom4te.cache test/runtest.sh pgreplay-$(VERSION).tar* 35 | 36 | tarball: distclean 37 | if tar --version 2>/dev/null | grep -q GNU; then : ;else echo "this requires GNU tar" 1>&2; exit 1; fi 38 | ln -s "`pwd`" /tmp/pgreplay-$(VERSION) 39 | tar -cf pgreplay-$(VERSION).tar -h -C /tmp --exclude .svn --exclude CVS --exclude pgreplay-$(VERSION).tar pgreplay-$(VERSION) --exclude test/testrun.c 40 | rm /tmp/pgreplay-$(VERSION) 41 | gzip -9 pgreplay-$(VERSION).tar 42 | 43 | $(EXE).html: $(EXE).1 44 | groff -Thtml -mman $< > $@ 45 | 46 | test: $(EXE) test/runtest.sh 47 | cd test && ./runtest.sh 48 | 49 | install: install_bin install_man 50 | 51 | install_bin: $(EXE) 52 | $(INSTALL) -d $(DESTDIR)$(bindir) 53 | $(INSTALL) -m 0755 $< $(DESTDIR)$(bindir)/$< 54 | 55 | install_man: $(EXE).1 56 | $(INSTALL) -d $(DESTDIR)$(mandir)/man1 57 | $(INSTALL) -m 0644 $< $(DESTDIR)$(mandir)/man1/$< 58 | 59 | install_html: $(EXE).html 60 | $(INSTALL) -d $(DESTDIR)$(htmldir) 61 | $(INSTALL) -m 0644 $< $(DESTDIR)$(htmldir)/$< 62 | 63 | .c.o: 64 | $(CC) -c $(CPPFLAGS) $(CFLAGS) -DVERSION='"$(VERSION)"' $< 65 | -------------------------------------------------------------------------------- /configure.in: -------------------------------------------------------------------------------- 1 | # -*- Autoconf -*- 2 | # Process this file with autoconf to produce a configure script. 3 | 4 | AC_PREREQ([2.62]) 5 | AC_INIT([pgreplay], [1.5.0], [https://github.com/laurenz/pgreplay/issues], [pgreplay], [http://laurenz.github.io/pgreplay/]) 6 | AC_CONFIG_SRCDIR([parse.c]) 7 | AC_CONFIG_HEADERS([config.h]) 8 | AC_CANONICAL_BUILD 9 | 10 | # Checks for programs. 11 | AC_PROG_INSTALL 12 | AC_PROG_CC([gcc cc]) 13 | # add -Wall to CFLAGS for gcc 14 | if test "$GCC" = yes; then CFLAGS="-Wall $CFLAGS"; fi 15 | 16 | # check if ld supports -rpath 17 | AC_MSG_CHECKING([whether the linker supports -rpath]) 18 | save_LDFLAGS="$LDFLAGS" 19 | LDFLAGS="$LDFLAGS -Wl,-rpath,/usr/lib" 20 | AC_LINK_IFELSE([AC_LANG_SOURCE([[main() {}]])], [rpath=yes], [rpath=no]) 21 | LDFLAGS="$save_LDFLAGS" 22 | AC_MSG_RESULT([$rpath]) 23 | 24 | AC_ARG_WITH([postgres], 25 | [AS_HELP_STRING([--with-postgres=DIR], 26 | [specify location of pg_config for your PostgreSQL installation])], 27 | [if ! test -x "$with_postgres"/pg_config; then 28 | echo "*** pg_config not found in '$with_postgres'" 1>&2 29 | exit 1 30 | fi 31 | pgbindir=`"$with_postgres"/pg_config --bindir` 32 | AC_SUBST([pgbindir]) 33 | if test $build_os = mingw32; then pglibdir="$pgbindir"; else pglibdir=`"$with_postgres"/pg_config --libdir`; fi 34 | pgincludedir=`"$with_postgres"/pg_config --includedir` 35 | LDFLAGS="$LDFLAGS -L$pglibdir" 36 | if test $rpath = yes; then LDFLAGS="$LDFLAGS -Wl,-rpath,$pglibdir"; fi 37 | CFLAGS="$CFLAGS -I$pgincludedir" 38 | PATH=`"$with_postgres"/pg_config --bindir`:$PATH]) 39 | 40 | # Install with PostgreSQL by default 41 | AC_PREFIX_PROGRAM(pg_config) 42 | 43 | # Checks for libraries. 44 | AC_CHECK_LIB([pq], [PQsendPrepare], [], [AC_MSG_ERROR([ 45 | *** cannot locate PostgreSQL client library 46 | *** required is version 8.0 or better 47 | ])]) 48 | 49 | # special Windows settings 50 | if test $build_os = mingw32; then 51 | AC_SUBST([EXTRA_OBJS], [windows.o]) 52 | # assume socks library is present 53 | # we have a problem using AC_CHECK_LIB because the function is __stdcall 54 | LIBS="$LIBS -lwsock32" 55 | fi 56 | 57 | # Checks for header files. 58 | AC_HEADER_TIME 59 | AC_CHECK_HEADERS([fcntl.h netinet/in.h stdint.h stdlib.h string.h sys/time.h unistd.h inttypes.h]) 60 | AC_CHECK_HEADER([libpq-fe.h], [], [AC_MSG_ERROR([PostgreSQL header files not found])], [/* dummy */]) 61 | 62 | # Checks for typedefs, structures, and compiler characteristics. 63 | AC_SYS_LARGEFILE 64 | AC_TYPE_SIZE_T 65 | AC_TYPE_SSIZE_T 66 | AC_TYPE_UINT64_T 67 | AC_TYPE_UINT32_T 68 | AC_CHECK_SIZEOF(unsigned int) 69 | AC_CHECK_SIZEOF(unsigned long) 70 | AC_CHECK_SIZEOF(unsigned short) 71 | AC_C_BIGENDIAN 72 | 73 | # Checks for library functions. 74 | AC_FUNC_STRTOD 75 | AC_CHECK_FUNCS([gettimeofday select setenv strchr strtoul]) 76 | 77 | AC_CONFIG_FILES([Makefile test/runtest.sh]) 78 | AC_OUTPUT 79 | chmod u+x test/runtest.sh 80 | -------------------------------------------------------------------------------- /CHANGELOG: -------------------------------------------------------------------------------- 1 | Version 1.4.0 2 | 3 | Enhancements: 4 | - Add a replay option "-n" that provides a "dry run" without actually 5 | connecting to the database. 6 | Patch by Manuel Vives. 7 | 8 | Bugfixes: 9 | - Fix Makefile to call "install" portably. 10 | It used to call "$(INSTALL) -D", which works on some, but not all systems 11 | (MAC OS X is an example where it doesn't work). 12 | Report by Jeff Doering. 13 | - Fix crash after replay of DEALLOCATE ALL. 14 | This caused a crash the next time prepared statements were deleted, for 15 | example at the end of the database session. 16 | Reported by Wan Shen Lim. 17 | 18 | Version 1.3.0 Feb 20 2017 19 | 20 | Enhancements: 21 | - Accept CPPFLAGS from configure in Makefile.in. 22 | Patch by Marko Kreen. 23 | - Add command line option -X to specify extra connect string fragment. 24 | Needed to specify unusual connect string options. 25 | Patch by Marko Kreen. 26 | - Introduce replay filter options with "-D database" and "-U username" 27 | to filter for a database or user during parsing. 28 | Patch by Gilles Darold. 29 | 30 | Version 1.2.0 Aug 17 2012 31 | 32 | Enhancements: 33 | - Introduce replay option "-j" to jump ahead when all connections are idle. 34 | This can speed up replay. The statistics will not include the skipped time, 35 | but delay warnings will work as expected. 36 | Idea and original patch by John Lumby. 37 | 38 | Bugfixes: 39 | - Fix failure to parse string constants like E'\\' where the backslash before 40 | a quotation mark is backslash-quoted. 41 | Bug discovered by Jeff Frost. 42 | 43 | Version 1.1.0 Feb 09 2012 44 | 45 | Enhancements: 46 | - Respect environment variable DESTDIR in Makefile for RPM packagers. 47 | - Improve execution delay reporting by introducing more intelligent time 48 | steps when a report is written; every 10 seconds is way too spammy. 49 | - Add documentation for interaction with pgFouine to the README. 50 | 51 | Bugfixes: 52 | - Fix incorrect assumption that everything that starts with a dollar 53 | sign is a dollar quoted string. This used to trigger bogus "end of dollar 54 | quote not found" error messages when $n placeholders are used in PREPARE 55 | statements. Discovered and fixed by Todd Owen. 56 | - When pgreplay waited for a response on a connection because it needed to 57 | send the next command, it used to sleep for one millisecond before 58 | polling the socket again. This proved to be too long on busy systems, 59 | where replay started to lag behind. Now pgreplay will not sleep, 60 | but keep polling until the response is there. 61 | 62 | Version 1.0.0 Jun 03 2011 63 | 64 | Bugfixes: 65 | - Fix a connection and memory leak introduced by the new handling of FATAL 66 | connection errors in 0.9.1. 67 | Discovered by Denis Kostin. 68 | 69 | Version 0.9.1 Feb 26 2011 70 | 71 | Enhancements: 72 | - Calculate parse and execution statistics and display them at the end 73 | of the run. 74 | 75 | Bugfixes: 76 | - Use "=" instead of "==" in "test" comparisons in configure. 77 | This improves portability. 78 | - Change replay file timestamp to seconds after midnight of 2000-01-01 79 | in local time. This makes the replay file format independent of time zones 80 | and avoids problems with mktime(3) implementations that don't like 81 | the UNIX epoch. 82 | - Ignore string literals in filter_bad_statements during log file parsing. 83 | This keeps the function from getting confused by the contents of the 84 | string. Discovered by Josh Berkus. 85 | - Correctly handle prepared statements without parameters. 86 | Discovered by Grigorij Lipin. 87 | - Fix a corner case bug in read_log_line that can cause data corruption 88 | when parsing a stderr log. Discovered by Grigorij Lipin. 89 | - Skip memory dumps in stderr log caused by "out of memory" errors 90 | instead of gagging on them. Discovered by Grigorij Lipin. 91 | - Don't gag if a connection attempt results in a FATAL error during replay. 92 | This can for example happen if max_connections has been exceeded or if a 93 | non-existant user is specified with "trust" authentication. 94 | Discovered by Grigorij Lipin. 95 | 96 | Version 0.9.0 Mar 19 2010 97 | 98 | - first release 99 | -------------------------------------------------------------------------------- /test/runtest.sh.in: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | pgbindir="@pgbindir@" 4 | if [ -z "$pgbindir" ]; then 5 | PSQL=psql 6 | else 7 | PSQL="$pgbindir"/psql 8 | fi 9 | 10 | echo "Testing stderr log parsing ... " 11 | ../pgreplay -f -o parse_err.rf -b '2010-12-31 10:59:52.241' -e '2010-12-31 11:00:30.005' errlog >parse_err.out 2>parse_err.err 12 | if [ $? -ne 0 ]; then 13 | echo "command failed, error messages:" 14 | cat parse_err.err 15 | rm -f parse_err.rf parse_err.out parse_err.err 16 | exit 1 17 | fi 18 | diff -u -b parse_err.err parse_err.exp >parse_err.diff 19 | if [ $? -ne 0 ]; then 20 | echo "unexpected messages, difference to expected:" 21 | cat parse_err.diff 22 | rm -f parse_err.rf parse_err.out parse_err.err parse_err.diff 23 | exit 1 24 | fi 25 | diff -u -b parse_err.out parse_err_out.exp >parse_err.diff 26 | if [ $? -ne 0 ]; then 27 | echo "unexpected output, difference to expected:" 28 | cat parse_err.diff 29 | rm -f parse_err.rf parse_err.out parse_err.err parse_err.diff 30 | exit 1 31 | fi 32 | cmp parse_err.rf replayfile >parse_err.diff 33 | if [ $? -ne 0 ]; then 34 | echo "unexpected replay file:" 35 | cat parse_err.diff 36 | rm -f parse_err.rf parse_err.out parse_err.err parse_err.diff 37 | exit 1 38 | fi 39 | echo "ok" 40 | rm -f parse_err.rf parse_err.out parse_err.err parse_err.diff 41 | 42 | echo "Testing CSV log parsing ... " 43 | ../pgreplay -f -c -o parse_csv.rf -b '2010-12-31 10:59:52.241' -e '2010-12-31 11:00:30.005' csvlog >parse_csv.out 2>parse_csv.err 44 | if [ $? -ne 0 ]; then 45 | echo "command failed, error messages:" 46 | cat parse_csv.err 47 | rm -f parse_csv.rf parse_csv.out parse_csv.err 48 | exit 1 49 | fi 50 | diff -u -b parse_csv.err parse_csv.exp >parse_csv.diff 51 | if [ $? -ne 0 ]; then 52 | echo "unexpected messages, difference to expected:" 53 | cat parse_csv.diff 54 | rm -f parse_csv.rf parse_csv.out parse_csv.err parse_csv.diff 55 | exit 1 56 | fi 57 | diff -u -b parse_csv.out parse_csv_out.exp >parse_csv.diff 58 | if [ $? -ne 0 ]; then 59 | echo "unexpected output, difference to expected:" 60 | cat parse_csv.diff 61 | rm -f parse_csv.rf parse_csv.out parse_csv.err parse_csv.diff 62 | exit 1 63 | fi 64 | cmp parse_csv.rf replayfile >parse_csv.diff 65 | if [ $? -ne 0 ]; then 66 | echo "unexpected replay file:" 67 | cat parse_csv.diff 68 | rm -f parse_csv.rf parse_csv.out parse_csv.err parse_csv.diff 69 | exit 1 70 | fi 71 | echo "ok" 72 | rm -f parse_csv.rf parse_csv.out parse_csv.err parse_csv.diff 73 | 74 | echo "Testing replay ... " 75 | "$PSQL" -U postgres -d postgres -l >/dev/null 2>&1 76 | if [ $? -ne 0 ]; then 77 | echo "skipped, cannot connect to database" 78 | echo "To run this test, configure your environment so that this command succeeds:" 79 | echo "\"$PSQL\" -U postgres -d postgres -l" 80 | exit 1 81 | fi 82 | ../pgreplay -r -s 2 -E UTF8 -d 1 replayfile 2>replay.err|sed '/^Replay statistics$/,$d' >replay.out 83 | if [ $? -ne 0 ]; then 84 | echo "command failed, error messages:" 85 | cat replay.err 86 | rm -f replay.out replay.err 87 | exit 1 88 | fi 89 | diff -u -b replay.out replay_out.exp >replay.diff 90 | if [ $? -ne 0 ]; then 91 | echo "unexpected output, difference to expected:" 92 | cat replay.diff 93 | rm -f replay.out replay.err replay.diff 94 | exit 1 95 | fi 96 | diff -u -b replay.err replay_err.exp >replay.diff 97 | if [ $? -ne 0 ]; then 98 | echo "unexpected messages, difference to expected:" 99 | cat replay.diff 100 | rm -f replay.out replay.err replay.diff 101 | exit 1 102 | fi 103 | echo "Testing dry run ... " 104 | ../pgreplay -r -n -E UTF8 replayfile 2>dryrun.err|sed '/calls per second/d' >dryrun.out 105 | if [ $? -ne 0 ]; then 106 | echo "command failed, error messages:" 107 | cat dryrun.err 108 | rm -f dryrun.out dryrun.err 109 | exit 1 110 | fi 111 | diff -u -b dryrun.out dryrun_out.exp >dryrun.diff 112 | if [ $? -ne 0 ]; then 113 | echo "unexpected output, difference to expected:" 114 | cat dryrun.diff 115 | rm -f dryrun.out dryrun.err dryrun.diff 116 | exit 1 117 | fi 118 | diff -u -b dryrun.err dryrun_err.exp >dryrun.diff 119 | if [ $? -ne 0 ]; then 120 | echo "unexpected messages, difference to expected:" 121 | cat dryrun.diff 122 | rm -f dryrun.out dryrun.err dryrun.diff 123 | exit 1 124 | fi 125 | 126 | echo "ok" 127 | rm -f replay.out replay.err replay.diff dryrun.out dryrun.err dryrun.diff 128 | exit 0 129 | -------------------------------------------------------------------------------- /config.h.in: -------------------------------------------------------------------------------- 1 | /* config.h.in. Generated from configure.in by autoheader. */ 2 | 3 | /* Define if building universal (internal helper macro) */ 4 | #undef AC_APPLE_UNIVERSAL_BUILD 5 | 6 | /* Define to 1 if you have the header file. */ 7 | #undef HAVE_FCNTL_H 8 | 9 | /* Define to 1 if you have the `gettimeofday' function. */ 10 | #undef HAVE_GETTIMEOFDAY 11 | 12 | /* Define to 1 if you have the header file. */ 13 | #undef HAVE_INTTYPES_H 14 | 15 | /* Define to 1 if you have the `pq' library (-lpq). */ 16 | #undef HAVE_LIBPQ 17 | 18 | /* Define to 1 if you have the header file. */ 19 | #undef HAVE_MEMORY_H 20 | 21 | /* Define to 1 if you have the header file. */ 22 | #undef HAVE_NETINET_IN_H 23 | 24 | /* Define to 1 if you have the `select' function. */ 25 | #undef HAVE_SELECT 26 | 27 | /* Define to 1 if you have the `setenv' function. */ 28 | #undef HAVE_SETENV 29 | 30 | /* Define to 1 if you have the header file. */ 31 | #undef HAVE_STDINT_H 32 | 33 | /* Define to 1 if you have the header file. */ 34 | #undef HAVE_STDLIB_H 35 | 36 | /* Define to 1 if you have the `strchr' function. */ 37 | #undef HAVE_STRCHR 38 | 39 | /* Define to 1 if you have the header file. */ 40 | #undef HAVE_STRINGS_H 41 | 42 | /* Define to 1 if you have the header file. */ 43 | #undef HAVE_STRING_H 44 | 45 | /* Define to 1 if you have the `strtoul' function. */ 46 | #undef HAVE_STRTOUL 47 | 48 | /* Define to 1 if you have the header file. */ 49 | #undef HAVE_SYS_STAT_H 50 | 51 | /* Define to 1 if you have the header file. */ 52 | #undef HAVE_SYS_TIME_H 53 | 54 | /* Define to 1 if you have the header file. */ 55 | #undef HAVE_SYS_TYPES_H 56 | 57 | /* Define to 1 if you have the header file. */ 58 | #undef HAVE_UNISTD_H 59 | 60 | /* Define to the address where bug reports for this package should be sent. */ 61 | #undef PACKAGE_BUGREPORT 62 | 63 | /* Define to the full name of this package. */ 64 | #undef PACKAGE_NAME 65 | 66 | /* Define to the full name and version of this package. */ 67 | #undef PACKAGE_STRING 68 | 69 | /* Define to the one symbol short name of this package. */ 70 | #undef PACKAGE_TARNAME 71 | 72 | /* Define to the home page for this package. */ 73 | #undef PACKAGE_URL 74 | 75 | /* Define to the version of this package. */ 76 | #undef PACKAGE_VERSION 77 | 78 | /* The size of `unsigned int', as computed by sizeof. */ 79 | #undef SIZEOF_UNSIGNED_INT 80 | 81 | /* The size of `unsigned long', as computed by sizeof. */ 82 | #undef SIZEOF_UNSIGNED_LONG 83 | 84 | /* The size of `unsigned short', as computed by sizeof. */ 85 | #undef SIZEOF_UNSIGNED_SHORT 86 | 87 | /* Define to 1 if you have the ANSI C header files. */ 88 | #undef STDC_HEADERS 89 | 90 | /* Define to 1 if you can safely include both and . */ 91 | #undef TIME_WITH_SYS_TIME 92 | 93 | /* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most 94 | significant byte first (like Motorola and SPARC, unlike Intel). */ 95 | #if defined AC_APPLE_UNIVERSAL_BUILD 96 | # if defined __BIG_ENDIAN__ 97 | # define WORDS_BIGENDIAN 1 98 | # endif 99 | #else 100 | # ifndef WORDS_BIGENDIAN 101 | # undef WORDS_BIGENDIAN 102 | # endif 103 | #endif 104 | 105 | /* Enable large inode numbers on Mac OS X 10.5. */ 106 | #ifndef _DARWIN_USE_64_BIT_INODE 107 | # define _DARWIN_USE_64_BIT_INODE 1 108 | #endif 109 | 110 | /* Number of bits in a file offset, on hosts where this is settable. */ 111 | #undef _FILE_OFFSET_BITS 112 | 113 | /* Define for large files, on AIX-style hosts. */ 114 | #undef _LARGE_FILES 115 | 116 | /* Define for Solaris 2.5.1 so the uint32_t typedef from , 117 | , or is not used. If the typedef were allowed, the 118 | #define below would cause a syntax error. */ 119 | #undef _UINT32_T 120 | 121 | /* Define for Solaris 2.5.1 so the uint64_t typedef from , 122 | , or is not used. If the typedef were allowed, the 123 | #define below would cause a syntax error. */ 124 | #undef _UINT64_T 125 | 126 | /* Define to `unsigned int' if does not define. */ 127 | #undef size_t 128 | 129 | /* Define to `int' if does not define. */ 130 | #undef ssize_t 131 | 132 | /* Define to the type of an unsigned integer type of width exactly 32 bits if 133 | such a type exists and the standard includes do not define it. */ 134 | #undef uint32_t 135 | 136 | /* Define to the type of an unsigned integer type of width exactly 64 bits if 137 | such a type exists and the standard includes do not define it. */ 138 | #undef uint64_t 139 | -------------------------------------------------------------------------------- /pgreplay.h: -------------------------------------------------------------------------------- 1 | #ifndef _PGREPLAY_H 2 | #define _PGREPLAY_H 1 3 | 4 | #include "config.h" 5 | 6 | /* safeguard against broken config.h */ 7 | #ifndef SIZEOF_UNSIGNED_INT 8 | # error SIZEOF_UNSIGNED_INT not defined in config.h. Please execute 'configure' first! 9 | #endif 10 | 11 | #if defined(WIN32) || defined(WIN64) 12 | # ifndef WINDOWS 13 | # define WINDOWS 14 | # endif 15 | #endif 16 | 17 | #include 18 | #include 19 | #include 20 | 21 | /* maximum length of a name in PostgreSQL */ 22 | #define NAMELEN 64 23 | 24 | /* types for replay items */ 25 | typedef enum { 26 | pg_connect = 0, 27 | pg_disconnect, 28 | pg_execute, 29 | pg_prepare, 30 | pg_exec_prepared, 31 | pg_cancel 32 | } replay_type; 33 | 34 | /* one "command" parsed from a log file to be replayed 35 | the definition is in replay_item.c */ 36 | typedef struct replay_item replay_item; 37 | 38 | typedef int (replay_item_provider_init)(const char *, int, const char *, const char *, const char *, const char *); 39 | typedef replay_item *(replay_item_provider)(); 40 | typedef void (replay_item_provider_finish)(); 41 | 42 | typedef int (replay_item_consumer_init)(const char *, const char *, int, const char *, double); 43 | typedef int (replay_item_consumer)(replay_item *); 44 | typedef void (replay_item_consumer_finish)(int); 45 | 46 | /* hash value for session ID is computed as low byte of background PID */ 47 | #define hash_session(x) (unsigned char)(x & 0xFF); 48 | 49 | /* printf/scanf formats for various data types */ 50 | #if SIZEOF_UNSIGNED_INT == 4 51 | # define UINT32_FORMAT "%x" 52 | #else 53 | # define UINT32_FORMAT "%hx" 54 | #endif 55 | 56 | #ifdef HAVE_INTTYPES_H 57 | # include 58 | # define UINT64_FORMAT "%" PRIx64 59 | #else 60 | /* fall back to guessing */ 61 | # ifdef WINDOWS 62 | # define UINT64_FORMAT "%I64x" 63 | # else 64 | # if SIZEOF_UNSIGNED_LONG == 8 65 | # define UINT64_FORMAT "%lx" 66 | # else 67 | # define UINT64_FORMAT "%llx" 68 | # endif 69 | # endif 70 | #endif 71 | 72 | /*********************/ 73 | /* defined in main.c */ 74 | /*********************/ 75 | 76 | extern int debug_level; 77 | 78 | /* destination of statistics output */ 79 | extern FILE *sf; 80 | 81 | /* if 1, backslash will escape the following single quote in string literal */ 82 | extern int backslash_quote; 83 | 84 | /* if 1, replay will skip idle intervals instead of sleeping */ 85 | extern int jump_enabled; 86 | 87 | /* extra connect options specified with the -X option */ 88 | extern char *extra_connstr; 89 | 90 | /* print debug messages */ 91 | #define debug(level, format, ...) { \ 92 | if (level <= debug_level) { \ 93 | fprintf (stderr, format, __VA_ARGS__); \ 94 | fflush(stderr); \ 95 | } \ 96 | } 97 | 98 | /***************************/ 99 | /* defined in replayitem.c */ 100 | /***************************/ 101 | 102 | /* functions to create replay items */ 103 | extern replay_item *replay_create_connect(const struct timeval *time, uint64_t session_id, const char *user, const char *database); 104 | extern replay_item *replay_create_disconnect(const struct timeval *time, uint64_t session_id); 105 | extern replay_item *replay_create_execute(const struct timeval *time, uint64_t session_id, const char *statement); 106 | extern replay_item *replay_create_prepare(const struct timeval *time, uint64_t session_id, const char *statement, const char *name); 107 | extern replay_item *replay_create_exec_prepared(const struct timeval *time, uint64_t session_id, const char *name, uint16_t count, char * const *values); 108 | extern replay_item *replay_create_cancel(const struct timeval *time, uint64_t session_id); 109 | 110 | /* free mamory of a replay_item */ 111 | extern void replay_free(replay_item *r); 112 | 113 | /* get attributes of a replay item */ 114 | extern replay_type replay_get_type(const replay_item *r); 115 | extern uint64_t replay_get_session_id(const replay_item *r); 116 | extern const struct timeval * replay_get_time(const replay_item *r); 117 | extern const char * replay_get_statement(const replay_item *r); 118 | extern const char * replay_get_name(const replay_item *r); 119 | extern const char * replay_get_user(const replay_item *r); 120 | extern const char * replay_get_database(const replay_item *r); 121 | extern int replay_get_valuecount(const replay_item *r); 122 | extern const char * const * replay_get_values(const replay_item *r); 123 | 124 | /* dump a replay item at debug level 3 */ 125 | extern void replay_print_debug(const replay_item *r); 126 | 127 | /* special replay_item that signals end-of-file */ 128 | extern replay_item * const end_item; 129 | 130 | /**********************/ 131 | /* defined in parse.c */ 132 | /**********************/ 133 | 134 | /* parse a timestamp (excluding time zone) */ 135 | extern const char * parse_time(const char *, struct timeval *); 136 | 137 | extern replay_item_provider parse_provider; 138 | extern replay_item_provider_init parse_provider_init; 139 | extern replay_item_provider_finish parse_provider_finish; 140 | 141 | /***************************/ 142 | /* defined in replayfile.c */ 143 | /***************************/ 144 | 145 | extern replay_item_provider file_provider; 146 | extern replay_item_provider_init file_provider_init; 147 | extern replay_item_provider_finish file_provider_finish; 148 | 149 | extern replay_item_consumer file_consumer; 150 | extern replay_item_consumer_init file_consumer_init; 151 | extern replay_item_consumer_finish file_consumer_finish; 152 | 153 | /*************************/ 154 | /* defined in database.c */ 155 | /*************************/ 156 | 157 | extern replay_item_consumer database_consumer; 158 | extern replay_item_consumer database_consumer_dry_run; 159 | extern replay_item_consumer_init database_consumer_init; 160 | extern replay_item_consumer_finish database_consumer_finish; 161 | 162 | #ifdef WINDOWS 163 | /************************/ 164 | /* defined in windows.c */ 165 | /************************/ 166 | 167 | extern void win_perror(const char *prefix, int is_network_error); 168 | #endif 169 | 170 | #endif 171 | -------------------------------------------------------------------------------- /pgreplay.1: -------------------------------------------------------------------------------- 1 | .TH pgreplay 1 "" "Jun 2011" "PostgreSQL Utilities" 2 | .SH NAME 3 | pgreplay \- PostgreSQL log file replayer for performance tests 4 | .SH SYNOPSIS 5 | \fBpgreplay\fP [\fIparse options\fR] [\fIreplay options\fR] 6 | [\fB-d\fR \fIlevel\fR] [\fIinfile\fR] 7 | .br 8 | \fBpgreplay\fP \fB-f\fP [\fIparse options\fR] [\fB-o\fP \fIoutfile\fR] 9 | [\fB-d\fR \fIlevel\fR] [\fIinfile\fR] 10 | .br 11 | \fBpgreplay\fP \fB-r\fP [\fIreplay options\fR] [\fB-d\fR \fIlevel\fR] 12 | [\fIinfile\fR] 13 | .SH DESCRIPTION 14 | \fBpgreplay\fR reads a PostgreSQL log file (\fInot\fR a WAL file), extracts 15 | the SQL statements and executes them in the same order and relative time 16 | against a PostgreSQL database cluster. 17 | A final report gives you a useful statistical analysis of your workload 18 | and its execution. 19 | .P 20 | In the first form, the log file \fIinfile\fR is replayed at the time it is 21 | read. 22 | .P 23 | With the \fB-f\fR option, \fBpgreplay\fR will not execute the statements, but 24 | write them to a \(oqreplay file\(cq \fIoutfile\fR that can be replayed with 25 | the third form. 26 | .P 27 | With the \fB-r\fP option, \fBpgreplay\fR will execute the statements in the 28 | replay file \fIinfile\fR that was created by the second form. 29 | .P 30 | If the execution of statements gets behind schedule, warning messages 31 | are issued that indicate that the server cannot handle the load in a 32 | timely fashion. 33 | The idea is to replay a real-world database workload as exactly as possible. 34 | .P 35 | To create a log file that can be parsed by \fBpgreplay\fR, you need to set the 36 | following parameters in \fBpostgresql.conf\fR: 37 | .IP 38 | \fBlog_min_messages=error\fR (or more) 39 | .br 40 | \fBlog_min_error_statement=log\fR (or more) 41 | .br 42 | \fBlog_connections=on\fR 43 | .br 44 | \fBlog_disconnections=on\fR 45 | .br 46 | \fBlog_line_prefix=\(aq%m|%u|%d|%c|\(aq\fR (if you don\(aqt use CSV logging) 47 | .br 48 | \fBlog_statement=\(aqall\(aq\fR 49 | .br 50 | \fBlc_messages\fR must be set to English (encoding does not matter) 51 | .br 52 | \fBbytea_output=escape\fR 53 | (from version 9.0 on, only if you want to replay the log on 8.4 or earlier) 54 | .P 55 | The database cluster against which you replay the SQL statements must be 56 | a clone of the database cluster that generated the logs from the time 57 | \fIimmediately before\fR the logs were generated. 58 | .P 59 | \fBpgreplay\fR is useful for performance tests, particularly in the following 60 | situations: 61 | .TP 4 62 | * 63 | You want to compare the performance of your PostgreSQL application 64 | on different hardware or different operating systems. 65 | .TP 4 66 | * 67 | You want to upgrade your database and want to make sure that the new 68 | database version does not suffer from performance regressions that 69 | affect you. 70 | .P 71 | Moreover, \fBpgreplay\fR can give you some feeling as to how your application 72 | \fImight\fR scale by allowing you to try to replay the workload at a higher 73 | speed. Be warned, though, that 500 users working at double speed is not really 74 | the same as 1000 users working at normal speed. 75 | .SH OPTIONS 76 | .SS Parse options: 77 | .TP 78 | \fB-c\fR 79 | Specifies that the log file is in \(aqcsvlog\(aq format (highly recommended) 80 | and not in \(aqstderr\(aq format. 81 | .TP 82 | \fB-b\fR \fItimestamp\fR 83 | Only log entries greater or equal to that timestamp will be parsed. 84 | The format is \fBYYYY-MM-DD HH:MM:SS.FFF\fR like in the log file. 85 | An optional time zone part will be ignored. 86 | .TP 87 | \fB-e\fR \fItimestamp\fR 88 | Only log entries less or equal to that timestamp will be parsed. 89 | The format is \fBYYYY-MM-DD HH:MM:SS.FFF\fR like in the log file. 90 | An optional time zone part will be ignored. 91 | .TP 92 | \fB-q\fR 93 | Specifies that a backslash in a simple string literal will escape 94 | the following single quote. 95 | This depends on configuration options like 96 | \fBstandard_conforming_strings\fR and is the default for server 97 | version 9.0 and less. 98 | .TP 99 | \fB-D\fR \fIdatabase\fR 100 | Only log entries related to the specified database will be parsed 101 | (this option can be specified multiple times for more than one database). 102 | .TP 103 | \fB-U\fR \fIusername\fR 104 | Only log entries related to the specified username will be parsed 105 | (this option can be specified multiple times for more than one user). 106 | .SS Replay options: 107 | .TP 108 | \fB-h\fR \fIhostname\fR 109 | Host name where the target database cluster is running (or directory where 110 | the UNIX socket can be found). Defaults to local connections. 111 | .br 112 | This works just like the \fB-h\fR option of \fBpsql\fR. 113 | .TP 114 | \fB-p\fR \fIport\fR 115 | TCP port where the target database cluster can be reached. 116 | .TP 117 | \fB-W\fR \fIpassword\fR 118 | By default, \fBpgreplay\fR assumes that the target database cluster 119 | is configured for \fItrust\fR authentication. With the \fB-W\fR option 120 | you can specify a password that will be used for all users in the cluster. 121 | .TP 122 | \fB-s\fR \fIfactor\fR 123 | Speed factor for replay, by default 1. This can be any valid positive 124 | floating point number. A \fIfactor\fR less than 1 will replay the workload 125 | in \(oqslow motion\(cq, while a \fIfactor\fR greater than 1 means 126 | \(oqfast forward\(cq. 127 | .TP 128 | \fB-E\fR \fIencoding\fR 129 | Specifies the encoding of the log file, which will be used as client 130 | encoding during replay. If it is omitted, your default client encoding will 131 | be used. 132 | .TP 133 | \fB-j\fR 134 | If all connections are idle, jump ahead to the next request instead of 135 | sleeping. This will speed up replay. Execution delays will still be reported 136 | correctly, but replay statistics will not contain the idle time. 137 | .TP 138 | \fB-n\fR 139 | Dry run mode. No connections to the server are made. 140 | Useful for checking if the replay file is corrupt or to get statistics 141 | about the replay file (number of statements, original duration, ...) 142 | .TP 143 | \fB-X\fR \fIoptions\fR 144 | Extra connection options for replay connections. These must be libpq 145 | connection options specified in the format \(oqoption=value [...]\(cq. 146 | .SS Output options: 147 | .TP 148 | \fB-o\fP \fIoutfile\fR 149 | specifies the replay file where the statements will be written 150 | for later replay. 151 | .SS Debug options: 152 | .TP 153 | \fB-d\fR \fIlevel\fR 154 | Specifies the trace level (between 1 and 3). Increasing levels will produce 155 | more detailed information about what \fBpgreplay\fR is doing. 156 | .TP 157 | \fB-v\fR 158 | Prints the program version and exits. 159 | .SH ENVIRONMENT 160 | .TP 161 | \fBPGHOST\fR 162 | Specifies the default value for the \fB-h\fR option. 163 | .TP 164 | \fBPGPORT\fR 165 | Specifies the default value for the \fB-p\fR option. 166 | .TP 167 | \fBPGCLIENTENCODING\fR 168 | Specifies the default value for the \fB-E\fR option. 169 | .SH LIMITATIONS 170 | \fBpgreplay\fR can only replay what is logged by PostgreSQL. 171 | This leads to some limitations: 172 | .TP 4 173 | * 174 | \fBCOPY\fR statements will not be replayed, because the copy data are not 175 | logged. 176 | .TP 4 177 | * 178 | Fast-path API function calls are not logged and will not be replayed. 179 | Unfortunately, this includes the Large Object API. 180 | .TP 4 181 | * 182 | Since the log file is always in the server encoding (which you can specify 183 | with the \fB-E\fR switch of \fBpgreplay\fR), all 184 | \fBSET client_encoding\fR statements will be ignored. 185 | .TP 4 186 | * 187 | Since the preparation time of prepared statements is not logged (unless 188 | \fBlog_min_messages\fR is \fBdebug2\fR or more), these statements will be 189 | prepared immediately before they are first executed during replay. 190 | .TP 4 191 | * 192 | Because the log file contains only text, query parameters and return values 193 | will always be in text and never in binary format. If you use binary mode to, 194 | say, transfer large binary data, \fBpgreplay\fR can cause significantly more 195 | network traffic than the original run. 196 | .TP 4 197 | * 198 | Sometimes, if a connection takes longer to complete, the session ID 199 | unexpectedly changes in the PostgreSQL log file. This causes \fBpgreplay\fR 200 | to treat the session as two different ones, resulting in an additional 201 | connection. This is arguably a bug in PostgreSQL. 202 | .SH AUTHOR 203 | Written by Laurenz Albe \fB\fR. 204 | -------------------------------------------------------------------------------- /test/replay_err.exp: -------------------------------------------------------------------------------- 1 | --------------------------- 2 | Item: time = 347108392.241000 3 | session id = 0x4d1db7a800004236 4 | type = connect 5 | user = postgres 6 | database = postgres 7 | --------------------------- 8 | --------------------------- 9 | Item: time = 347108392.241000 10 | session id = 0x4d1db7a800004236 11 | type = execute 12 | statement = 13 | --------------------------- 14 | --------------------------- 15 | Item: time = 347108392.241000 16 | session id = 0x4d1db7a800004227 17 | type = connect 18 | user = postgres 19 | database = postgres 20 | --------------------------- 21 | --------------------------- 22 | Item: time = 347108392.243000 23 | session id = 0x4d1db7a800004227 24 | type = execute 25 | statement = 26 | --------------------------- 27 | --------------------------- 28 | Item: time = 347108392.744000 29 | session id = 0x4d1db7a800004236 30 | type = execute 31 | statement = CREATE TABLE runtest( 32 | id integer PRIMARY KEY, 33 | c text NOT NULL, 34 | t timestamp with time zone NOT NULL, 35 | b bytea 36 | ) 37 | --------------------------- 38 | --------------------------- 39 | Item: time = 347108393.308000 40 | session id = 0x4d1db7a800004227 41 | type = prepare 42 | name = 43 | statement = INSERT INTO runtest (id, c, t, b) VALUES ($1, $2, $3, $4) 44 | --------------------------- 45 | --------------------------- 46 | Item: time = 347108393.308000 47 | session id = 0x4d1db7a800004227 48 | type = exec_prepared 49 | name = 50 | $1 = 1 51 | $2 = ein '€" 52 | $3 = 2008-08-08 10:30:00+00 53 | $4 = a\000b\011c 54 | --------------------------- 55 | --------------------------- 56 | Item: time = 347108393.500000 57 | session id = 0x4d5d5122000088b9 58 | type = connect 59 | user = hansi 60 | database = postgres 61 | --------------------------- 62 | --------------------------- 63 | Item: time = 347108393.827000 64 | session id = 0x4d1db7a900004242 65 | type = connect 66 | user = postgres 67 | database = postgres 68 | --------------------------- 69 | --------------------------- 70 | Item: time = 347108393.829000 71 | session id = 0x4d1db7a900004242 72 | type = execute 73 | statement = 74 | --------------------------- 75 | --------------------------- 76 | Item: time = 347108394.330000 77 | session id = 0x4d1db7a800004227 78 | type = prepare 79 | name = 80 | statement = SELECT 42 81 | --------------------------- 82 | --------------------------- 83 | Item: time = 347108394.330000 84 | session id = 0x4d1db7a800004227 85 | type = exec_prepared 86 | name = 87 | --------------------------- 88 | --------------------------- 89 | Item: time = 347108395.348000 90 | session id = 0x4d1db7a900004242 91 | type = prepare 92 | name = 93 | statement = INSERT INTO runtest (id, c, t, b) VALUES ($1, $2, $3, $4) 94 | --------------------------- 95 | --------------------------- 96 | Item: time = 347108395.348000 97 | session id = 0x4d1db7a900004242 98 | type = exec_prepared 99 | name = 100 | $1 = 2 101 | $2 = Zeile 102 | zwei 103 | $3 = 1968-10-20 11:00:00+00 104 | $4 = (null) 105 | --------------------------- 106 | --------------------------- 107 | Item: time = 347108395.866000 108 | session id = 0x4d1db7a800004236 109 | type = execute 110 | statement = 111 | --------------------------- 112 | --------------------------- 113 | Item: time = 347108396.868000 114 | session id = 0x4d1db7a800004227 115 | type = execute 116 | statement = BEGIN 117 | --------------------------- 118 | --------------------------- 119 | Item: time = 347108397.870000 120 | session id = 0x4d1db7a800004227 121 | type = prepare 122 | name = einf"ug 123 | statement = INSERT INTO runtest (id, c, t, b) VALUES ($1, $2, $3, $4) 124 | --------------------------- 125 | --------------------------- 126 | Item: time = 347108397.870000 127 | session id = 0x4d1db7a800004227 128 | type = exec_prepared 129 | name = einf"ug 130 | $1 = 6 131 | $2 = mit Tabulator 132 | $3 = 2050-03-31 22:00:00+00 133 | $4 = (null) 134 | --------------------------- 135 | --------------------------- 136 | Item: time = 347108398.372000 137 | session id = 0x4d1db7a900004242 138 | type = prepare 139 | name = 140 | statement = INSERT INTO runtest (id, c, t, b) VALUES ($1, $2, $3, $4) 141 | --------------------------- 142 | --------------------------- 143 | Item: time = 347108398.372000 144 | session id = 0x4d1db7a900004242 145 | type = exec_prepared 146 | name = 147 | $1 = 6 148 | $2 = irgendein Mist 149 | $3 = 1999-12-31 23:00:00+00 150 | $4 = \000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000... 151 | --------------------------- 152 | --------------------------- 153 | Item: time = 347108399.889000 154 | session id = 0x4d1db7a800004227 155 | type = exec_prepared 156 | name = einf"ug 157 | $1 = 7 158 | $2 = 159 | $3 = 2004-02-29 22:59:59.999999+00 160 | $4 = (null) 161 | --------------------------- 162 | --------------------------- 163 | Item: time = 347108400.390000 164 | session id = 0x4d1db7a800004227 165 | type = execute 166 | statement = COMMIT 167 | --------------------------- 168 | --------------------------- 169 | Item: time = 347108401.414000 170 | session id = 0x4d1db7a800004227 171 | type = execute 172 | statement = DEALLOCATE "einf""ug" 173 | --------------------------- 174 | --------------------------- 175 | Item: time = 347108401.934000 176 | session id = 0x4d1db7a800004236 177 | type = execute 178 | statement = SELECT id, c, t, b FROM runtest WHERE id <= 4 179 | --------------------------- 180 | --------------------------- 181 | Item: time = 347108402.438000 182 | session id = 0x4d1db7a900004242 183 | type = execute 184 | statement = BEGIN; DECLARE mycur CURSOR FOR SELECT * FROM runtest ORDER BY id; DEALLOCATE/* a silly /* nested */ comment */ 185 | -- another comment 186 | ALL 187 | --------------------------- 188 | --------------------------- 189 | Item: time = 347108403.440000 190 | session id = 0x4d1db7a900004242 191 | type = execute 192 | statement = MOVE FORWARD 5 FROM mycur 193 | --------------------------- 194 | --------------------------- 195 | Item: time = 347108403.941000 196 | session id = 0x4d1db7a900004242 197 | type = execute 198 | statement = FETCH mycur 199 | --------------------------- 200 | --------------------------- 201 | Item: time = 347108404.443000 202 | session id = 0x4d1db7a900004242 203 | type = execute 204 | statement = select proname, oid from pg_catalog.pg_proc where proname in ('lo_open', 'lo_close', 'lo_creat', 'lo_create', 'lo_unlink', 'lo_lseek', 'lo_tell', 'lo_truncate', 'loread', 'lowrite') and pronamespace = (select oid from pg_catalog.pg_namespace where nspname = 'pg_catalog') 205 | --------------------------- 206 | --------------------------- 207 | Item: time = 347108404.946000 208 | session id = 0x4d1db7a900004242 209 | type = execute 210 | statement = COMMIT 211 | --------------------------- 212 | --------------------------- 213 | Item: time = 347108405.983000 214 | session id = 0x4d1db7a800004227 215 | type = execute 216 | statement = SELECT pg_sleep(22) 217 | --------------------------- 218 | --------------------------- 219 | Item: time = 347108427.985000 220 | session id = 0x4d1db7a800004227 221 | type = execute 222 | statement = SELECT pg_sleep(10) 223 | --------------------------- 224 | --------------------------- 225 | Item: time = 347108428.486000 226 | session id = 0x4d1db7a800004227 227 | type = cancel 228 | --------------------------- 229 | --------------------------- 230 | Item: time = 347108429.489000 231 | session id = 0x4d1db7a800004236 232 | type = execute 233 | statement = DROP TABLE runtest 234 | --------------------------- 235 | --------------------------- 236 | Item: time = 347108430.005000 237 | session id = 0x4d1db7a800004236 238 | type = disconnect 239 | --------------------------- 240 | --------------------------- 241 | Item: time = 347108430.005000 242 | session id = 0x4d1db7a800004227 243 | type = disconnect 244 | --------------------------- 245 | --------------------------- 246 | Item: time = 347108430.005000 247 | session id = 0x4d1db7a900004242 248 | type = disconnect 249 | --------------------------- 250 | --------------------------- 251 | Item: time = 347108430.005000 252 | session id = 0x4d5d5122000088b9 253 | type = disconnect 254 | --------------------------- 255 | -------------------------------------------------------------------------------- /replayitem.c: -------------------------------------------------------------------------------- 1 | #include "pgreplay.h" 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | /* type for one replay line */ 10 | struct replay_item { 11 | struct timeval time; 12 | uint64_t session_id; 13 | replay_type type; 14 | uint16_t count; 15 | char **data; 16 | }; 17 | 18 | /* special replay_item that signals end-of-file */ 19 | static replay_item end_replay_item = {{0, 0}, 0, -1, 0, NULL}; 20 | replay_item * const end_item = &end_replay_item; 21 | 22 | /* create common part of a replay_item */ 23 | static replay_item *replay_create(const struct timeval *time, uint64_t session_id, replay_type type, uint16_t count) { 24 | replay_item *r; 25 | 26 | r = malloc(sizeof(struct replay_item)); 27 | if (NULL == r) { 28 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)sizeof(struct replay_item)); 29 | return NULL; 30 | } 31 | r->time.tv_sec = time->tv_sec; 32 | r->time.tv_usec = time->tv_usec; 33 | r->session_id = session_id; 34 | r->type = type; 35 | r->count = count; 36 | if (0 == count) { 37 | r->data = NULL; 38 | } else { 39 | r->data = calloc(count, sizeof(char *)); 40 | if (NULL == r->data) { 41 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)count * sizeof(char *)); 42 | free(r); 43 | return NULL; 44 | } 45 | } 46 | 47 | return r; 48 | } 49 | 50 | replay_item *replay_create_connect(const struct timeval *time, uint64_t session_id, const char *user, const char *database) { 51 | replay_item *r; 52 | 53 | debug(3, "Entering replay_create_connect%s\n", ""); 54 | 55 | r = replay_create(time, session_id, pg_connect, 2); 56 | if (NULL == r) { 57 | return NULL; 58 | } 59 | 60 | (r->data)[0] = malloc(strlen(user) + 1); 61 | if (NULL == (r->data)[0]) { 62 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)strlen(user) + 1); 63 | free(r->data); 64 | free(r); 65 | return NULL; 66 | } 67 | strcpy((r->data)[0], user); 68 | 69 | (r->data)[1] = malloc(strlen(database) + 1); 70 | if (NULL == (r->data)[1]) { 71 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)strlen(user) + 1); 72 | free((r->data)[0]); 73 | free(r->data); 74 | free(r); 75 | return NULL; 76 | } 77 | strcpy((r->data)[1], database); 78 | 79 | debug(3, "Leaving replay_create_connect%s\n", ""); 80 | return r; 81 | } 82 | 83 | replay_item *replay_create_disconnect(const struct timeval *time, uint64_t session_id) { 84 | replay_item *r; 85 | 86 | debug(3, "Entering replay_create_disconnect%s\n", ""); 87 | 88 | r = replay_create(time, session_id, pg_disconnect, 0); 89 | if (NULL == r) { 90 | return NULL; 91 | } 92 | debug(3, "Leaving replay_create_disconnect%s\n", ""); 93 | return r; 94 | } 95 | 96 | replay_item *replay_create_execute(const struct timeval *time, uint64_t session_id, const char *statement) { 97 | replay_item *r; 98 | 99 | debug(3, "Entering replay_create_execute%s\n", ""); 100 | 101 | r = replay_create(time, session_id, pg_execute, 1); 102 | if (NULL == r) { 103 | return NULL; 104 | } 105 | 106 | (r->data)[0] = malloc(strlen(statement) + 1); 107 | if (NULL == (r->data)[0]) { 108 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)strlen(statement) + 1); 109 | free(r->data); 110 | free(r); 111 | return NULL; 112 | } 113 | strcpy((r->data)[0], statement); 114 | 115 | debug(3, "Leaving replay_create_execute%s\n", ""); 116 | return r; 117 | } 118 | 119 | replay_item *replay_create_prepare(const struct timeval *time, uint64_t session_id, const char *name, const char *statement) { 120 | replay_item *r; 121 | 122 | debug(3, "Entering replay_create_prepare%s\n", ""); 123 | 124 | r = replay_create(time, session_id, pg_prepare, 2); 125 | if (NULL == r) { 126 | return NULL; 127 | } 128 | 129 | (r->data)[0] = malloc(strlen(statement) + 1); 130 | if (NULL == (r->data)[0]) { 131 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)strlen(statement) + 1); 132 | free(r->data); 133 | free(r); 134 | return NULL; 135 | } 136 | strcpy((r->data)[0], statement); 137 | 138 | (r->data)[1] = malloc(strlen(name) + 1); 139 | if (NULL == (r->data)[1]) { 140 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)strlen(name) + 1); 141 | free((r->data)[0]); 142 | free(r->data); 143 | free(r); 144 | return NULL; 145 | } 146 | strcpy((r->data)[1], name); 147 | 148 | debug(3, "Leaving replay_create_prepare%s\n", ""); 149 | return r; 150 | } 151 | 152 | replay_item *replay_create_exec_prepared(const struct timeval *time, uint64_t session_id, const char*name, uint16_t count, char * const *values) { 153 | replay_item *r; 154 | int i; 155 | 156 | debug(3, "Entering replay_create_exec_prepared%s\n", ""); 157 | 158 | r = replay_create(time, session_id, pg_exec_prepared, count + 1); 159 | if (NULL == r) { 160 | return NULL; 161 | } 162 | 163 | (r->data)[0] = malloc(strlen(name) + 1); 164 | if (NULL == (r->data)[0]) { 165 | free(r->data); 166 | free(r); 167 | return NULL; 168 | } 169 | strcpy((r->data)[0], name); 170 | 171 | for (i=1; idata)[i] = malloc(strlen(values[i-1]) + 1); 174 | if (NULL == (r->data)[i]) { 175 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)strlen(values[i-1]) + 1); 176 | for (--i; i>=0; --i) { 177 | if ((r->data)[i]) { 178 | free((r->data)[i]); 179 | } 180 | } 181 | free(r->data); 182 | free(r); 183 | return NULL; 184 | } 185 | strcpy((r->data)[i], values[i-1]); 186 | } else { 187 | (r->data)[i] = NULL; 188 | } 189 | } 190 | 191 | debug(3, "Leaving replay_create_exec_prepared%s\n", ""); 192 | return r; 193 | } 194 | 195 | replay_item *replay_create_cancel(const struct timeval *time, uint64_t session_id) { 196 | replay_item *r; 197 | 198 | debug(3, "Entering replay_create_cancel%s\n", ""); 199 | 200 | r = replay_create(time, session_id, pg_cancel, 0); 201 | if (NULL == r) { 202 | return NULL; 203 | } 204 | debug(3, "Leaving replay_create_cancel%s\n", ""); 205 | return r; 206 | } 207 | 208 | void replay_free(replay_item *r) { 209 | int i; 210 | 211 | debug(3, "Entering replay_free%s\n", ""); 212 | 213 | assert((pg_connect == r->type) || (pg_disconnect == r->type) || (pg_execute == r->type) || (pg_prepare == r->type) || (pg_exec_prepared == r->type) || (pg_cancel == r->type)); 214 | 215 | for (i=0; icount; ++i) { 216 | if ((r->data)[i]) { 217 | free((r->data)[i]); 218 | } 219 | } 220 | if (r->count) { 221 | free(r->data); 222 | } 223 | free(r); 224 | 225 | debug(3, "Leaving replay_free%s\n", ""); 226 | } 227 | 228 | replay_type replay_get_type(const replay_item *r) { 229 | return r->type; 230 | } 231 | 232 | uint64_t replay_get_session_id(const replay_item *r) { 233 | return r->session_id; 234 | } 235 | 236 | const struct timeval * replay_get_time(const replay_item *r) { 237 | return &(r->time); 238 | } 239 | 240 | const char * replay_get_statement(const replay_item *r) { 241 | assert((pg_execute == r->type) || (pg_prepare == r->type)); 242 | 243 | return (r->data)[0]; 244 | } 245 | 246 | const char * replay_get_name(const replay_item *r) { 247 | assert((pg_prepare == r->type) || (pg_exec_prepared == r->type)); 248 | 249 | return (pg_prepare == r->type) ? (r->data)[1] : (r->data)[0]; 250 | } 251 | 252 | const char * replay_get_user(const replay_item *r) { 253 | assert(pg_connect == r->type); 254 | 255 | return (r->data)[0]; 256 | } 257 | 258 | const char * replay_get_database(const replay_item *r) { 259 | assert(pg_connect == r->type); 260 | 261 | return (r->data)[1]; 262 | } 263 | 264 | int replay_get_valuecount(const replay_item *r) { 265 | assert(pg_exec_prepared == r->type); 266 | 267 | return r->count - 1; 268 | } 269 | 270 | const char * const * replay_get_values(const replay_item *r) { 271 | assert(pg_exec_prepared == r->type); 272 | 273 | return (const char * const *)((r->data) + 1); 274 | } 275 | 276 | /* maximal part of a value for display */ 277 | #define SAMPLE_SIZE 100 278 | 279 | void replay_print_debug(const replay_item *r) { 280 | replay_type type; 281 | int i; 282 | char valuepart[SAMPLE_SIZE+4], *p; 283 | 284 | valuepart[SAMPLE_SIZE] = '.'; 285 | valuepart[SAMPLE_SIZE+1] = '.'; 286 | valuepart[SAMPLE_SIZE+2] = '.'; 287 | valuepart[SAMPLE_SIZE+3] = '\0'; 288 | 289 | debug(1, "---------------------------%s\n", ""); 290 | debug(1, "Item: time = %lu.%06lu\n", (unsigned long)r->time.tv_sec, (unsigned long)r->time.tv_usec); 291 | debug(1, " session id = 0x" UINT64_FORMAT "\n", r->session_id); 292 | type = r->type; 293 | debug(1, " type = %s\n", 294 | (pg_connect == type) ? "connect" : 295 | ((pg_disconnect == type) ? "disconnect" : 296 | ((pg_execute == type) ? "execute" : 297 | ((pg_prepare == type) ? "prepare" : 298 | ((pg_exec_prepared == type) ? "exec_prepared" : 299 | ((pg_cancel == type) ? "cancel" : "unknown") 300 | ) 301 | ) 302 | ) 303 | ) 304 | ); 305 | switch (type) { 306 | case pg_connect: 307 | debug(1, " user = %s\n", replay_get_user(r)); 308 | debug(1, " database = %s\n", replay_get_database(r)); 309 | case pg_disconnect: 310 | case pg_cancel: 311 | break; 312 | case pg_prepare: 313 | debug(1, " name = %s\n", replay_get_name(r)); 314 | case pg_execute: 315 | debug(1, " statement = %s\n", replay_get_statement(r)); 316 | break; 317 | case pg_exec_prepared: 318 | debug(1, " name = %s\n", replay_get_name(r)); 319 | for (i=0; i 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #ifdef WINDOWS 10 | # include 11 | # include 12 | # define FILE_MODE S_IRUSR | S_IWUSR 13 | #else 14 | # ifdef HAVE_NETINET_IN_H 15 | # include 16 | # endif 17 | # define FILE_MODE S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH 18 | #endif 19 | #include 20 | #include 21 | #include 22 | 23 | /* input or output file */ 24 | static int filed=0; 25 | 26 | /* functions to convert 64-bit integers between host and network byte order */ 27 | #ifndef htonll 28 | # ifdef WORDS_BIGENDIAN 29 | # define htonll(x) (x) 30 | # define ntohll(x) (x) 31 | # else 32 | # define htonll(x) ((((uint64_t)htonl(x)) << 32) + htonl(x >> 32)) 33 | # define ntohll(x) ((((uint64_t)ntohl(x)) << 32) + ntohl(x >> 32)) 34 | # endif 35 | #endif 36 | 37 | /* this length indicates a null value */ 38 | #define NULL_VALUE 0x80000000 39 | 40 | /* wrapper functions for read and write */ 41 | static int do_write(const void * const buf, size_t count) { 42 | int rc = write(filed, buf, count); 43 | 44 | if (-1 == rc) { 45 | perror("Error writing to output file"); 46 | return 0; 47 | } else if (count != rc) { 48 | fprintf(stderr, "Error: not all bytes written to output file\n"); 49 | return 0; 50 | } 51 | 52 | return 1; 53 | } 54 | 55 | static int do_read(void *buf, size_t count, int *eof_indicator) { 56 | int rc = read(filed, buf, count); 57 | 58 | if (eof_indicator) { 59 | *eof_indicator = 0; 60 | } 61 | 62 | if (-1 == rc) { 63 | perror("Error reading from input file"); 64 | return 0; 65 | } else if (eof_indicator && (0 == rc)) { 66 | *eof_indicator = 1; 67 | } else if (count != rc) { 68 | fprintf(stderr, "Error: unexpected end of file on input file\n"); 69 | return 0; 70 | } 71 | 72 | return 1; 73 | } 74 | 75 | /* write a string to the output file */ 76 | static int write_string(char const * const s) { 77 | uint32_t u32, len; 78 | 79 | /* write length + NULL indicator (4 byte) */ 80 | if (NULL == s) { 81 | len = NULL_VALUE; 82 | } else { 83 | len = strlen(s); 84 | } 85 | u32 = htonl(len); 86 | if (! do_write(&u32, 4)) { 87 | return 0; 88 | } else if (NULL != s) { 89 | /* write string */ 90 | if (! do_write(s, len)) { 91 | return 0; 92 | } 93 | } 94 | 95 | return 1; 96 | } 97 | 98 | /* malloc and read a string from the input file */ 99 | static int read_string(char ** const s) { 100 | uint32_t u32, len; 101 | 102 | /* read length (4 byte) */ 103 | if (! do_read(&u32, 4, NULL)) { 104 | return 0; 105 | } 106 | len = ntohl(u32); 107 | if (NULL_VALUE == len) { 108 | *s = NULL; 109 | } else { 110 | /* allocate the string */ 111 | if (! (*s = malloc(len + 1))) { 112 | fprintf(stderr, "Cannot allocate %d bytes of memory\n", len + 1); 113 | return 0; 114 | } else { 115 | /* read string */ 116 | if (! do_read(*s, len, NULL)) { 117 | return 0; 118 | } 119 | (*s)[len] = '\0'; 120 | } 121 | } 122 | 123 | return 1; 124 | } 125 | 126 | int file_provider_init(const char *infile, int cvs, const char *begin_time, const char *end_time, const char *db_only, const char *usr_only) { 127 | int rc = 1; 128 | debug(3, "Entering file_provider_init%s\n", ""); 129 | 130 | if (NULL == infile) { 131 | filed = 0; 132 | #ifdef WINDOWS 133 | setmode(filed, O_BINARY); 134 | #endif 135 | } else { 136 | if (-1 == (filed = open(infile, O_RDONLY 137 | #ifdef WINDOWS 138 | | O_BINARY 139 | #endif 140 | ))) { 141 | perror("Error opening input file:"); 142 | rc = 0; 143 | } 144 | } 145 | 146 | debug(3, "Leaving file_provider_init%s\n", ""); 147 | 148 | return rc; 149 | } 150 | 151 | void file_provider_finish() { 152 | debug(3, "Entering file_provider_finish%s\n", ""); 153 | 154 | if (0 != filed) { 155 | if (close(filed)) { 156 | perror("Error closing input file:"); 157 | } 158 | } 159 | 160 | debug(3, "Leaving file_provider_finish%s\n", ""); 161 | } 162 | 163 | replay_item * file_provider() { 164 | replay_item *r = NULL; 165 | uint16_t u16; 166 | uint32_t u32; 167 | uint64_t u64, session_id = 0; 168 | struct timeval tv; 169 | replay_type type = -1; 170 | int ok = 1, i = 0, eof; 171 | unsigned long count; 172 | char *user, *database, *statement, *name, **values, nl; 173 | 174 | debug(3, "Entering file_provider%s\n", ""); 175 | 176 | /* read timestamp (8 byte) */ 177 | if (! do_read(&u32, 4, &eof)) { 178 | ok = 0; 179 | } else { 180 | /* handle expected end-of-file condition */ 181 | if (eof) { 182 | return end_item; 183 | } 184 | 185 | tv.tv_sec = ntohl(u32); 186 | if (! do_read(&u32, 4, NULL)) { 187 | ok = 0; 188 | } else { 189 | tv.tv_usec = ntohl(u32); 190 | } 191 | } 192 | 193 | /* read session_id (8 byte) */ 194 | if (ok && do_read(&u64, 8, NULL)) { 195 | session_id = ntohll(u64); 196 | } else { 197 | ok = 0; 198 | } 199 | 200 | /* read type (1 byte) */ 201 | if (ok) { 202 | u16 = 0; 203 | if (! do_read((char *)(&u16) + 1, 1, NULL)) { 204 | ok = 0; 205 | } else { 206 | type = ntohs(u16); 207 | if ((type < pg_connect) || (type > pg_cancel)) { 208 | fprintf(stderr, "Error: unknown type %u encountered\n", type); 209 | ok = 0; 210 | } 211 | } 212 | } 213 | 214 | /* read type specific stuff */ 215 | if (ok) { 216 | switch (type) { 217 | case pg_connect: 218 | if (read_string(&user)) { 219 | if (read_string(&database)) { 220 | r = replay_create_connect(&tv, session_id, user, database); 221 | free(database); 222 | } 223 | free(user); 224 | } 225 | break; 226 | case pg_disconnect: 227 | r = replay_create_disconnect(&tv, session_id); 228 | break; 229 | case pg_execute: 230 | if (read_string(&statement)) { 231 | r = replay_create_execute(&tv, session_id, statement); 232 | free(statement); 233 | } 234 | break; 235 | case pg_prepare: 236 | if (read_string(&statement)) { 237 | if (read_string(&name)) { 238 | r = replay_create_prepare(&tv, session_id, name, statement); 239 | free(name); 240 | } 241 | free(statement); 242 | } 243 | break; 244 | case pg_exec_prepared: 245 | /* read statement name */ 246 | if (read_string(&name)) { 247 | /* number of bind arguments (2 byte) */ 248 | if (do_read(&u16, 2, NULL)) { 249 | count = ntohs(u16); 250 | if (NULL == (values = calloc(count, sizeof(char *)))) { 251 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", count * sizeof(char *)); 252 | } else { 253 | /* read bind values */ 254 | while (i < count) { 255 | if (read_string(values + i)) { 256 | ++i; 257 | } else { 258 | break; 259 | } 260 | } 261 | if (i == count) { 262 | r = replay_create_exec_prepared(&tv, session_id, name, count, values); 263 | } 264 | while (--i >= 0) { 265 | if (values[i]) { 266 | free(values[i]); 267 | } 268 | } 269 | free(values); 270 | } 271 | } 272 | free(name); 273 | } 274 | break; 275 | case pg_cancel: 276 | r = replay_create_cancel(&tv, session_id); 277 | break; 278 | } 279 | } 280 | 281 | /* read new-line at the end of the record */ 282 | if (r && do_read(&nl, 1, NULL) && ('\n' != nl)) { 283 | fprintf(stderr, "Error: missing new-line at end of line\n"); 284 | if (r) { 285 | replay_free(r); 286 | r = NULL; 287 | } 288 | } 289 | 290 | if (r && (1 <= debug_level) && (end_item != r)) { 291 | replay_print_debug(r); 292 | } 293 | 294 | debug(3, "Leaving file_provider%s\n", ""); 295 | 296 | return r; 297 | } 298 | 299 | int file_consumer_init(const char *outfile, const char *host, int port, const char *passwd, double factor) { 300 | debug(3, "Entering file_consumer_init%s\n", ""); 301 | 302 | if ((NULL == outfile) || ('\0' == outfile[0]) 303 | || (('-' == outfile[0]) && ('\0' == outfile[1]))) { 304 | filed = 1; 305 | #ifdef WINDOWS 306 | /* set stdout to binary mode */ 307 | setmode(filed, O_BINARY); 308 | #endif 309 | } else { 310 | if (-1 == (filed = open(outfile, O_WRONLY | O_CREAT | O_TRUNC 311 | #ifdef WINDOWS 312 | | O_BINARY 313 | #endif 314 | , FILE_MODE))) { 315 | perror("Error opening output file:"); 316 | return 0; 317 | } 318 | } 319 | 320 | debug(3, "Leaving file_consumer_init%s\n", ""); 321 | return 1; 322 | } 323 | 324 | void file_consumer_finish(int dry_run) { 325 | debug(3, "Entering file_consumer_finish%s\n", ""); 326 | 327 | if (1 != filed) { 328 | if (close(filed)) { 329 | perror("Error closing output file:"); 330 | } 331 | } 332 | 333 | debug(3, "Leaving file_consumer_finish%s\n", ""); 334 | } 335 | 336 | int file_consumer(replay_item *item) { 337 | const struct timeval *tv = replay_get_time(item); 338 | uint16_t count; 339 | const replay_type type = replay_get_type(item); 340 | uint16_t u16, i; 341 | uint32_t u32; 342 | uint64_t u64; 343 | int rc = 1; 344 | const char * const *values; 345 | 346 | debug(3, "Entering file_consumer%s\n", ""); 347 | 348 | /* write timestamp (8 byte) */ 349 | u32 = htonl(tv->tv_sec); 350 | if (! do_write(&u32, 4)) { 351 | rc = -1; 352 | } else { 353 | u32 = htonl(tv->tv_usec); 354 | if (! do_write(&u32, 4)) { 355 | rc = -1; 356 | } 357 | } 358 | 359 | /* write session_id (8 byte) */ 360 | if (1 == rc) { 361 | u64 = htonll(replay_get_session_id(item)); 362 | if (! do_write(&u64, 8)) { 363 | rc = -1; 364 | } 365 | } 366 | 367 | /* write type (1 byte) */ 368 | if (1 == rc) { 369 | u16 = htons((uint16_t) type); 370 | if (! do_write((char *)(&u16) + 1, 1)) { 371 | rc = -1; 372 | } 373 | } 374 | 375 | /* write type specific stuff */ 376 | if (1 == rc) { 377 | switch (type) { 378 | case pg_connect: 379 | if (! write_string(replay_get_user(item))) { 380 | rc = -1; 381 | } else if (! write_string(replay_get_database(item))) { 382 | rc = -1; 383 | } 384 | break; 385 | case pg_disconnect: 386 | break; 387 | case pg_execute: 388 | if (! write_string(replay_get_statement(item))) { 389 | rc = -1; 390 | } 391 | break; 392 | case pg_prepare: 393 | if (! write_string(replay_get_statement(item))) { 394 | rc = -1; 395 | } else if (! write_string(replay_get_name(item))) { 396 | rc = -1; 397 | } 398 | break; 399 | case pg_exec_prepared: 400 | count = replay_get_valuecount(item); 401 | /* write statement name */ 402 | if (! write_string(replay_get_name(item))) { 403 | rc = -1; 404 | } else { 405 | /* write count (2 byte) */ 406 | u16 = htons(count); 407 | if (! do_write(&u16, 2)) { 408 | rc = -1; 409 | } else { 410 | /* write values */ 411 | values = replay_get_values(item); 412 | for (i=0; i 4 | #include 5 | #include 6 | #include 7 | #include 8 | #ifdef WINDOWS 9 | # include 10 | #endif 11 | 12 | /* from getopt */ 13 | extern char *optarg; 14 | 15 | int debug_level = 0; 16 | 17 | /* destination of statistics output */ 18 | FILE *sf; 19 | 20 | /* if 1, backslash will escape the following single quote in string literal */ 21 | int backslash_quote = 0; 22 | 23 | /* if 1, replay will skip idle intervals instead of sleeping */ 24 | int jump_enabled = 0; 25 | 26 | /* extra connect options specified with the -X option */ 27 | char *extra_connstr; 28 | 29 | /* wrapper for setenv, returns 0 on success and -1 on error */ 30 | static int do_setenv(const char *name, const char *value) { 31 | int rc; 32 | 33 | #ifdef WINDOWS 34 | if (0 == SetEnvironmentVariable(name, value)) { 35 | win_perror("Error setting environment variable", 0); 36 | rc = -1; 37 | } else { 38 | rc = 0; 39 | } 40 | #else 41 | if (-1 == (rc = setenv(name, value, 1))) { 42 | fprintf(stderr, "Error setting environment variable\n"); 43 | } 44 | #endif 45 | 46 | return rc; 47 | } 48 | 49 | static void version(FILE *f) { 50 | fprintf(f, "pgreplay %s\n", VERSION); 51 | } 52 | 53 | static void help(FILE *f) { 54 | fprintf(f, "\n"); 55 | version(f); 56 | fprintf(f, "==============\n"); 57 | fprintf(f, "\nUsage: pgreplay [] [] []\n"); 58 | fprintf(f, " pgreplay -f [] [-o ] []\n"); 59 | fprintf(f, " pgreplay -r [] []\n\n"); 60 | fprintf(f, " The first form parses a PostgreSQL log file and replays the\n"); 61 | fprintf(f, "statements against a database.\n"); 62 | fprintf(f, " The second form parses a PostgreSQL log file and writes the\n"); 63 | fprintf(f, "contents to a \"replay file\" that can be replayed with -r.\n"); 64 | fprintf(f, " The third form replays a file generated with -f.\n\n"); 65 | fprintf(f, "Parse options:\n"); 66 | fprintf(f, " -c (assume CSV logfile)\n"); 67 | fprintf(f, " -b (start time for parsing logfile)\n"); 68 | fprintf(f, " -e (end time for parsing logfile)\n"); 69 | fprintf(f, " -q ( \\' in string literal is a single quote)\n\n"); 70 | fprintf(f, " -D (database name to use as filter for parsing logfile)\n"); 71 | fprintf(f, " -U (username to use as filter for parsing logfile)\n"); 72 | fprintf(f, "Replay options:\n"); 73 | fprintf(f, " -h \n"); 74 | fprintf(f, " -p \n"); 75 | fprintf(f, " -W (must be the same for all users)\n"); 76 | fprintf(f, " -s (speed factor for replay)\n"); 77 | fprintf(f, " -E (server encoding)\n"); 78 | fprintf(f, " -j (skip idle time during replay)\n"); 79 | fprintf(f, " -X (extra libpq connect options)\n\n"); 80 | fprintf(f, " -n (dry-run, will replay file without running queries)\n\n"); 81 | fprintf(f, "Debugging:\n"); 82 | fprintf(f, " -d (level between 1 and 3)\n"); 83 | fprintf(f, " -v (prints version and exits)\n"); 84 | } 85 | 86 | int main(int argc, char **argv) { 87 | int arg, parse_only = 0, replay_only = 0, port = -1, csv = 0, 88 | parse_opt = 0, replay_opt = 0, rc = 0, dry_run = 0; 89 | double factor = 1.0; 90 | char *host = NULL, *encoding = NULL, *endptr, *passwd = NULL, 91 | *outfilename = NULL, *infilename = NULL, 92 | *database_only = NULL, *username_only = NULL, 93 | start_time[24] = { '\0' }, end_time[24] = { '\0' }; 94 | const char *errmsg; 95 | unsigned long portnr = 0l, debug = 0l, length; 96 | replay_item_provider *provider; 97 | replay_item_provider_init *provider_init; 98 | replay_item_provider_finish *provider_finish; 99 | replay_item_consumer *consumer; 100 | replay_item_consumer_init *consumer_init; 101 | replay_item_consumer_finish *consumer_finish; 102 | replay_item *item = NULL; 103 | 104 | /* initialize errno to avoid bogus error messages */ 105 | errno = 0; 106 | 107 | /* parse arguments */ 108 | opterr = 0; 109 | while (-1 != (arg = getopt(argc, argv, "vfro:h:p:W:s:E:d:cb:e:qjnX:D:U:"))) { 110 | switch (arg) { 111 | case 'v': 112 | version(stdout); 113 | return 0; 114 | break; 115 | case 'f': 116 | parse_only = 1; 117 | if (replay_only) { 118 | fprintf(stderr, "Error: options -p and -r are mutually exclusive\n"); 119 | help(stderr); 120 | return 1; 121 | } 122 | break; 123 | case 'r': 124 | replay_only = 1; 125 | if (parse_only) { 126 | fprintf(stderr, "Error: options -p and -r are mutually exclusive\n"); 127 | help(stderr); 128 | return 1; 129 | } 130 | break; 131 | case 'o': 132 | outfilename = ('\0' == *optarg) ? NULL : optarg; 133 | break; 134 | case 'h': 135 | replay_opt = 1; 136 | 137 | host = ('\0' == *optarg) ? NULL : optarg; 138 | break; 139 | case 'p': 140 | replay_opt = 1; 141 | 142 | portnr = strtoul(optarg, &endptr, 0); 143 | if (('\0' == *optarg) || ('\0' != *endptr)) { 144 | fprintf(stderr, "Not a valid port number: \"%s\"\n", optarg); 145 | help(stderr); 146 | return 1; 147 | } 148 | if ((portnr < 1) || (65535 < portnr)) { 149 | fprintf(stderr, "Port number must be between 1 and 65535\n"); 150 | help(stderr); 151 | return 1; 152 | } 153 | port = (int)portnr; 154 | break; 155 | case 'W': 156 | replay_opt = 1; 157 | 158 | passwd = ('\0' == *optarg) ? NULL : optarg; 159 | break; 160 | case 's': 161 | replay_opt = 1; 162 | 163 | factor = strtod(optarg, &endptr); 164 | if (('\0' == *optarg) || ('\0' != *endptr)) { 165 | fprintf(stderr, "Not a valid floating point number: \"%s\"\n", optarg); 166 | help(stderr); 167 | return 1; 168 | } 169 | if (0 != errno) { 170 | perror("Error converting speed factor"); 171 | help(stderr); 172 | return 1; 173 | } 174 | if (factor <= 0.0) { 175 | fprintf(stderr, "Factor must be greater than 0\n"); 176 | help(stderr); 177 | return 1; 178 | } 179 | break; 180 | case 'E': 181 | replay_opt = 1; 182 | 183 | encoding = ('\0' == *optarg) ? NULL : optarg; 184 | break; 185 | case 'd': 186 | debug = strtoul(optarg, &endptr, 0); 187 | if (('\0' == *optarg) || ('\0' != *endptr)) { 188 | fprintf(stderr, "Not a valid debug level: \"%s\"\n", optarg); 189 | help(stderr); 190 | return 1; 191 | } 192 | if ((debug < 0) || (3 < debug)) { 193 | fprintf(stderr, "Debug level must be between 0 and 3\n"); 194 | help(stderr); 195 | return 1; 196 | } 197 | debug_level = (int)debug; 198 | break; 199 | case 'c': 200 | parse_opt = 1; 201 | 202 | csv = 1; 203 | break; 204 | case 'b': 205 | parse_opt = 1; 206 | 207 | if (NULL == (errmsg = parse_time(optarg, NULL))) { 208 | strncpy(start_time, optarg, 23); 209 | } else { 210 | fprintf(stderr, "Error in begin timestamp: %s\n", errmsg); 211 | help(stderr); 212 | return 1; 213 | } 214 | break; 215 | case 'e': 216 | parse_opt = 1; 217 | 218 | if (NULL == (errmsg = parse_time(optarg, NULL))) { 219 | strncpy(end_time, optarg, 23); 220 | } else { 221 | fprintf(stderr, "Error in end timestamp: %s\n", errmsg); 222 | help(stderr); 223 | return 1; 224 | } 225 | break; 226 | case 'q': 227 | backslash_quote = 1; 228 | break; 229 | case 'j': 230 | replay_opt = 1; 231 | 232 | jump_enabled = 1; 233 | break; 234 | case 'n': 235 | replay_opt = 1; 236 | 237 | dry_run = 1; 238 | break; 239 | case 'X': 240 | replay_opt = 1; 241 | 242 | extra_connstr = optarg; 243 | break; 244 | case 'D': 245 | parse_opt = 1; 246 | 247 | if (NULL == database_only) { 248 | length = strlen(optarg) + 3; 249 | database_only = malloc(length); 250 | if (NULL != database_only) 251 | strcpy(database_only, "\\"); 252 | } else { 253 | length = strlen(database_only) + strlen(optarg) + 2; 254 | database_only = realloc(database_only, length); 255 | } 256 | if (NULL == database_only) { 257 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", length); 258 | return 1; 259 | } 260 | 261 | strcat(database_only, optarg); 262 | strcat(database_only, "\\"); 263 | break; 264 | case 'U': 265 | parse_opt = 1; 266 | 267 | if (NULL == username_only) { 268 | length = strlen(optarg) + 3; 269 | username_only = malloc(length); 270 | if (NULL != username_only) 271 | strcpy(username_only, "\\"); 272 | } else { 273 | length = strlen(username_only) + strlen(optarg) + 2; 274 | username_only = realloc(username_only, length); 275 | } 276 | if (NULL == username_only) { 277 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", length); 278 | return 1; 279 | } 280 | 281 | strcat(username_only, optarg); 282 | strcat(username_only, "\\"); 283 | break; 284 | case '?': 285 | if (('?' == optopt) || ('h' == optopt)) { 286 | help(stdout); 287 | return 0; 288 | } else { 289 | fprintf(stderr, "Error: unknown option -%c\n", optopt); 290 | help(stderr); 291 | return 1; 292 | } 293 | break; 294 | } 295 | } 296 | 297 | if (optind + 1 < argc) { 298 | fprintf(stderr, "More than one argument given\n"); 299 | help(stderr); 300 | return 1; 301 | } 302 | 303 | if (optind + 1 == argc) { 304 | infilename = argv[optind]; 305 | } 306 | 307 | if (parse_only && replay_opt) { 308 | fprintf(stderr, "Error: cannot specify replay option with -f\n"); 309 | help(stderr); 310 | return 1; 311 | } 312 | 313 | if (replay_only && parse_opt) { 314 | fprintf(stderr, "Error: cannot specify parse option with -r\n"); 315 | help(stderr); 316 | return 1; 317 | } 318 | 319 | if (NULL != outfilename) { 320 | if (! parse_only) { 321 | fprintf(stderr, "Error: option -o is only allowed with -f\n"); 322 | help(stderr); 323 | return 1; 324 | } 325 | } 326 | 327 | /* set default encoding */ 328 | if (NULL != encoding) { 329 | if (-1 == do_setenv("PGCLIENTENCODING", encoding)) { 330 | return 1; 331 | } 332 | } 333 | 334 | /* figure out destination for statistics output */ 335 | if (parse_only && (NULL == outfilename)) { 336 | sf = stderr; /* because replay file will go to stdout */ 337 | } else { 338 | sf = stdout; 339 | } 340 | 341 | /* configure main loop */ 342 | 343 | if (replay_only) { 344 | provider_init = &file_provider_init; 345 | provider = &file_provider; 346 | provider_finish = &file_provider_finish; 347 | } else { 348 | provider_init = &parse_provider_init; 349 | provider = &parse_provider; 350 | provider_finish = &parse_provider_finish; 351 | } 352 | 353 | if (parse_only) { 354 | consumer_init = &file_consumer_init; 355 | consumer_finish = &file_consumer_finish; 356 | consumer = &file_consumer; 357 | } else { 358 | consumer_init = &database_consumer_init; 359 | consumer_finish = &database_consumer_finish; 360 | if (0 == dry_run) { 361 | consumer = &database_consumer; 362 | } else { 363 | consumer = &database_consumer_dry_run; 364 | } 365 | } 366 | 367 | /* main loop */ 368 | 369 | if (! (*provider_init)( 370 | infilename, 371 | csv, 372 | (('\0' == start_time[0]) ? NULL : start_time), 373 | (('\0' == end_time[0]) ? NULL : end_time), 374 | database_only, 375 | username_only 376 | )) 377 | { 378 | rc = 1; 379 | } 380 | 381 | if ((0 == rc) && (*consumer_init)(outfilename, host, port, passwd, factor)) { 382 | /* try to get first item */ 383 | if (! (item = (*provider)())) { 384 | rc = 1; 385 | } 386 | } else { 387 | rc = 1; 388 | } 389 | 390 | while ((0 == rc) && (end_item != item)) { 391 | switch ((*consumer)(item)) { 392 | case 0: /* item not consumed */ 393 | break; 394 | case 1: /* item consumed */ 395 | if (! (item = (*provider)())) { 396 | rc = 1; 397 | } 398 | break; 399 | default: /* error occurred */ 400 | rc = 1; 401 | } 402 | } 403 | 404 | /* no statistics output if there was an error */ 405 | if (1 == rc) { 406 | sf = NULL; 407 | } 408 | 409 | (*provider_finish)(); 410 | (*consumer_finish)(dry_run); 411 | 412 | return rc; 413 | } 414 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | pgreplay - record and replay real-life database workloads 2 | ========================================================= 3 | 4 | pgreplay reads a PostgreSQL log file (*not* a WAL file), extracts the 5 | SQL statements and executes them in the same order and with the original 6 | timing against a PostgreSQL database. 7 | 8 | If the execution of statements gets behind schedule, warning messages 9 | are issued that indicate that the server cannot handle the load in a 10 | timely fashion. 11 | 12 | A final report gives you a useful statistical analysis of your workload 13 | and its execution. 14 | 15 | The idea is to replay a real-world database workload as exactly as possible. 16 | 17 | This is useful for performance tests, particularly in the following 18 | situations: 19 | - You want to compare the performance of your PostgreSQL application 20 | on different hardware or different operating systems. 21 | - You want to upgrade your database and want to make sure that the new 22 | database version does not suffer from performance regressions that 23 | affect you. 24 | 25 | Moreover, pgreplay can give you some feeling as to how your application 26 | *might* scale by allowing you to try to replay the workload at a higher 27 | speed (if that is possible; see 28 | [implementation details](#implementation-details) below). 29 | Be warned, though, that 500 users working at double speed is not really 30 | the same as 1000 users working at normal speed. 31 | 32 | While pgreplay will find out if your database application will encounter 33 | performance problems, it does not provide a lot of help in the analysis of 34 | the cause of these problems. Combine pgreplay with a specialized analysis 35 | program like [pgBadger](https://pgbadger.darold.net/) for that. 36 | 37 | As an additional feature, pgreplay lets you split the replay in two 38 | parts: you can parse the log file and create a "replay file", which 39 | contains just the statements to be replayed and is hopefully much 40 | smaller than the original log file. 41 | Such a replay file can then be run against a database. 42 | 43 | pgreplay is written by Laurenz Albe and is inspired by "Playr" 44 | which never made it out of Beta. 45 | 46 | Installation 47 | ============ 48 | 49 | pgreplay needs PostgreSQL 8.0 or better. 50 | 51 | It is supposed to compile without warnings and run on all platforms 52 | supported by PostgreSQL. 53 | Since I only got to test it on Linux, AIX, FreeBSD and Windows, there may be 54 | problems with other platforms. I am interested in reports and fixes for 55 | these platforms. 56 | On Windows, only the MinGW build environment is supported (I have no 57 | other compiler). That means that there is currently no 64-bit build 58 | for Windows (but a 32-bit executable should work fine anywhere). 59 | 60 | To build pgreplay, you will need the `pg_config` utility. If you installed 61 | PostgreSQL using installation packages, you will probably have to install 62 | the development package that contains `pg_config` and the header files. 63 | 64 | If `pg_config` is on the `PATH`, the installation process will look like this: 65 | 66 | - unpack the tarball 67 | - `./configure` 68 | - `make` 69 | - `make test` (optional, described below) 70 | - `make install` (as superuser) 71 | 72 | If your PostgreSQL installation is in a nonstandard directory, you 73 | will have to use the `--with-postgres=` 74 | option of `configure`. 75 | 76 | Unless you link it statically, pgreplay requires the PostgreSQL client 77 | shared library on the system where it is run. 78 | 79 | The following utilities are only necessary if you intend to develop pgreplay: 80 | - autoconf 2.62 or better to generate `configure` 81 | - GNU tar to `make tarball` (unless you want to roll it by hand) 82 | - groff to make the HTML documentation with `make html` 83 | 84 | Homebrew (macOS) 85 | ---------------- 86 | 87 | You can install pgreplay via Homebrew using this repository as a tap: 88 | 89 | ``` 90 | brew tap laurenz/pgreplay https://github.com/laurenz/pgreplay 91 | brew install laurenz/pgreplay/pgreplay 92 | ``` 93 | 94 | After installation, verify the binary with `pgreplay -v`. 95 | 96 | Docker 97 | ------ 98 | 99 | The `Dockerfile` provided with the software can be used as a starting 100 | point for creating a container that runs pgreplay. Adapt is as necessary. 101 | 102 | Here are commands to build and run the container: 103 | 104 | ``` 105 | # build the image 106 | docker build -t laurenz/pgreplay -f Dockerfile . 107 | 108 | # and run it 109 | docker run --rm -ti -v $(pwd):/app -w /app laurenz/pgreplay pgreplay -h 110 | ``` 111 | 112 | Testing 113 | ------- 114 | 115 | You can run a test on pgreplay before installing by running `make test`. 116 | This will parse sample log files and check that the result is as 117 | expected. 118 | 119 | Then an attempt is made to replay the log files and check if that 120 | works as expected. For this you need psql installed and a PostgreSQL server 121 | running (on this or another machine) so that the following command 122 | will succeed: 123 | 124 | psql -U postgres -d postgres -l 125 | 126 | You can set up the `PGPORT` and `PGHOST` environment variables and a password 127 | file for the user if necessary. 128 | 129 | There have to be a login roles named `hansi` and `postgres` in the database, 130 | and both users must be able to connect without a password. Only `postgres` 131 | will be used to run actual SQL statements. The regression test will create 132 | a table `runtest` and use it, and it will drop the table when it is done. 133 | 134 | Usage 135 | ===== 136 | 137 | First, you will need to record your real-life workload. 138 | For that, set the following parameters in `postgresql.conf`: 139 | 140 | - `log_min_messages = error` (or more) 141 | (if you know that you have no cancel requests, `log` will do) 142 | - `log_min_error_statement = log` (or more) 143 | - `log_connections = on` 144 | - `log_disconnections = on` 145 | - `log_line_prefix = '%m|%u|%d|%c|'` (if you don't use CSV logging) 146 | - `log_statement = 'all'` 147 | - `lc_messages` must be set to English (the encoding does not matter) 148 | - `bytea_output = escape` (from version 9.0 on, only if you want to replay 149 | the log on 8.4 or earlier) 150 | 151 | It is highly recommended that you use CSV logging, because anything that 152 | the PostgreSQL server or any loaded modules write to standard error will 153 | be written to the stderr log and might confuse the parser. 154 | 155 | Then let your users have their way with the database. 156 | 157 | Make sure that you have a `pg_dumpall` of the database cluster from the time 158 | of the start of your log file (or use the `-b` option with the time of your 159 | backup). Alternatively, you can use point-in-time-recovery to clone your 160 | database at the appropriate time. 161 | 162 | When you are done, restore the database (in the "before" state) to the 163 | machine where you want to perform the load test and run pgreplay against 164 | that database. 165 | 166 | Try to create a scenario as similar to your production system as 167 | possible (except for the change you want to test, of course). For example, 168 | if your clients connect over the network, run pgreplay on a different 169 | machine from where the database server is running. 170 | 171 | Since passwords are not logged (and pgreplay consequently has no way of 172 | knowing them), you have two options: either change `pg_hba.conf` on the 173 | test database to allow `trust` authentication or (if that is unacceptable) 174 | create a password file as described by the PostgreSQL documentation. 175 | Alternatively, you can change the passwords of all application users 176 | to one single password that you supply to pgreplay with the `-W` option. 177 | 178 | Limitations 179 | =========== 180 | 181 | pgreplay can only replay what is logged by PostgreSQL. 182 | This leads to some limitations: 183 | 184 | - `COPY` statements will not be replayed, because the copy data are not logged. 185 | I could have supported `COPY TO` statements, but that would have imposed a 186 | requirement that the directory structure on the replay system must be 187 | identical to the original machine. 188 | And if your application runs on the same machine as your database and they 189 | interact on the file system, pgreplay will probably not help you much 190 | anyway. 191 | - Fast-path API function calls are not logged and will not be replayed. 192 | Unfortunately, this includes the Large Object API. 193 | - Since the log file is always written in the database encoding (which you 194 | can specify with the `-E` switch of pgreplay), all `SET client_encoding` 195 | statements will be ignored. 196 | - If your cluster contains databases with different encoding, the log file 197 | will have mixed encoding as well. You cannot use pgreplay well in such 198 | an environment, because many statements against databases whose 199 | encoding does not match the `-E` switch will fail. 200 | - Since the preparation time of prepared statements is not logged (unless 201 | `log_min_messages` is `debug2` or more), these statements will be prepared 202 | immediately before they are first executed during replay. 203 | - All parameters of prepared statements are logged as strings, no matter 204 | what type was originally specified during bind. 205 | This can cause errors during replay with expressions like `$1 + $2`, 206 | which will cause the error `operator is not unique: unknown + unknown`. 207 | 208 | While pgreplay makes sure that commands are sent to the server in the 209 | order in which they were originally executed, there is no way to guarantee 210 | that they will be executed in the same order during replay: Network 211 | delay, processor contention and other factors may cause a later command 212 | to "overtake" an earlier one. While this does not matter if the 213 | commands don't affect each other, it can lead to SQL statements hitting 214 | locks unexpectedly, causing replay to deadlock and "hang". 215 | This is particularly likely if many different sessions change the same data 216 | repeatedly in short intervals. 217 | 218 | You can work around this problem by canceling the waiting statement with 219 | pg_cancel_backend. Replay should continue normally after that. 220 | 221 | Implementation details 222 | ====================== 223 | 224 | pgreplay will track the "session ID" associated with each log entry (the 225 | session ID uniquely identifies a database connection). 226 | For each new session ID, a new database connection will be opened during 227 | replay. Each statement will be sent on the corresponding connection, so 228 | transactions are preserved and concurrent sessions cannot get in each 229 | other's way. 230 | 231 | The order of statements in the log file is strictly preserved, so there 232 | cannot be any race conditions caused by different execution speeds on 233 | separate connections. On the other hand, that means that long running 234 | queries on one connection may stall execution on concurrent connections, 235 | but that's all you can get if you want to reproduce the exact same 236 | workload on a system that behaves differently. 237 | 238 | As an example, consider this (simplified) log file: 239 | 240 | session 1|connect 241 | session 2|connect 242 | session 1|statement: BEGIN 243 | session 1|statement: SELECT something(1) 244 | session 2|statement: BEGIN 245 | session 2|statement: SELECT something(2) 246 | session 1|statement: SELECT something(3) 247 | session 2|statement: ROLLBACK 248 | session 2|disconnect 249 | session 1|statement: COMMIT 250 | session 2|disconnect 251 | 252 | This will cause two database connections to be opened, so the `ROLLBACK` in 253 | session 2 will not affect session 1. 254 | If `SELECT something(2)` takes longer than expected (longer than it did in 255 | the original), that will not stall the execution of `SELECT something(3)` 256 | because it runs on a different connection. The `ROLLBACK`, however, has to 257 | wait for the completion of the long statement. Since the order of statements 258 | is preserved, the `COMMIT` on session 1 will have to wait until the `ROLLBACK` 259 | on session 2 has started (but it does not have to wait for the completion of 260 | the `ROLLBACK`). 261 | 262 | pgreplay is implemented in C and makes heavy use of asynchronous command 263 | processing (which is the reason why it is implemented in C). 264 | This way a single process can handle many concurrent connections, which 265 | makes it possible to get away without multithreading or multiprocessing. 266 | 267 | This avoids the need for synchronization and many portability problems. 268 | But since TINSTAAFL, the choice of C brings along its own portability 269 | problems. Go figure. 270 | 271 | Replay file format 272 | ------------------ 273 | 274 | The replay file is a binary file, integer numbers are stored in network 275 | byte order. 276 | 277 | Each record in the replay file corresponds to one database operation 278 | and is constructed as follows: 279 | - 4-byte `unsigned int`: log file timestamp in seconds since 2000-01-01 280 | - 4-byte `unsigned int`: fractional part of log file timestamp in microseconds 281 | - 8-byte `unsigned int`: session id 282 | - 1-byte `unsigned int`: type of the database action: 283 | - 0 is connect 284 | - 1 is disconnect 285 | - 2 is simple statement execution 286 | - 3 is statement preparation 287 | - 4 is execution of a prepared statement 288 | - 5 is cancel request 289 | - The remainder of the record is specific to the action, strings are stored 290 | with a preceeding 4-byte unsigned int that contains the length. 291 | Read the source for details. 292 | - Each record is terminated by a new-line character (byte 0x0A). 293 | 294 | Support 295 | ======= 296 | 297 | If you have a problem or question, the preferred option is to [open an 298 | issue](https://github.com/laurenz/pgreplay/issues). 299 | This requires a GitHub account. 300 | 301 | Professional support can be bought from 302 | [CYBERTEC PostgreSQL International GmbH](https://www.cybertec-postgresql.com/). 303 | 304 | TODO list 305 | ========= 306 | 307 | Nothing currently. Tell me if you have good ideas. 308 | -------------------------------------------------------------------------------- /pgreplay.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 6 | 7 | 8 | 9 | 10 | 16 | pgreplay 17 | 18 | 19 | 20 | 21 |

pgreplay

22 | 23 | NAME
24 | SYNOPSIS
25 | DESCRIPTION
26 | OPTIONS
27 | ENVIRONMENT
28 | LIMITATIONS
29 | AUTHOR
30 | 31 |
32 | 33 | 34 |

NAME 35 | 36 |

37 | 38 | 39 |

pgreplay 40 | − PostgreSQL log file replayer for performance 41 | tests

42 | 43 |

SYNOPSIS 44 | 45 |

46 | 47 | 48 | 49 |

pgreplay 50 | [parse options] [replay options] [-d 51 | level] [infile]
52 | pgreplay -f [parse options] [-o 53 | outfile] [-d level] [infile] 54 |
55 | pgreplay -r [replay options] [-d 56 | level] [infile]

57 | 58 |

DESCRIPTION 59 | 60 |

61 | 62 | 63 | 64 |

pgreplay 65 | reads a PostgreSQL log file (not a WAL file), 66 | extracts the SQL statements and executes them in the same 67 | order and relative time against a PostgreSQL database 68 | cluster. A final report gives you a useful statistical 69 | analysis of your workload and its execution.

70 | 71 |

In the first 72 | form, the log file infile is replayed at the time it 73 | is read.

74 | 75 |

With the 76 | -f option, pgreplay will not execute the 77 | statements, but write them to a ‘replay file’ 78 | outfile that can be replayed with the third form.

79 | 80 |

With the 81 | -r option, pgreplay will execute the 82 | statements in the replay file infile that was created 83 | by the second form.

84 | 85 |

If the 86 | execution of statements gets behind schedule, warning 87 | messages are issued that indicate that the server cannot 88 | handle the load in a timely fashion. The idea is to replay a 89 | real-world database workload as exactly as possible.

90 | 91 |

To create a log 92 | file that can be parsed by pgreplay, you need to set 93 | the following parameters in postgresql.conf:

94 | 95 | 96 |

log_min_messages=error 97 | (or more)
98 | log_min_error_statement=log (or more)
99 | log_connections=on
100 | log_disconnections=on
101 | log_line_prefix='%m|%u|%d|%c|' (if you don't use CSV 102 | logging)
103 | log_statement='all'
104 | lc_messages must be set to English (encoding does not 105 | matter)
106 | bytea_output=escape (from version 9.0 on, only if you 107 | want to replay the log on 8.4 or earlier)

108 | 109 |

The database 110 | cluster against which you replay the SQL statements must be 111 | a clone of the database cluster that generated the logs from 112 | the time immediately before the logs were 113 | generated.

114 | 115 | 116 |

pgreplay 117 | is useful for performance tests, particularly in the 118 | following situations:

119 | 120 | 122 | 123 | 124 | 128 | 129 | 135 | 136 | 137 | 141 | 142 | 148 |
125 | 126 | 127 |

*

 130 | 131 | 132 |

You want to compare the performance of your PostgreSQL 133 | application on different hardware or different operating 134 | systems.

138 | 139 | 140 |

*

 143 | 144 | 145 |

You want to upgrade your database and want to make sure 146 | that the new database version does not suffer from 147 | performance regressions that affect you.

149 | 150 |

Moreover, 151 | pgreplay can give you some feeling as to how your 152 | application might scale by allowing you to try to 153 | replay the workload at a higher speed. Be warned, though, 154 | that 500 users working at double speed is not really the 155 | same as 1000 users working at normal speed.

156 | 157 |

OPTIONS 158 | 159 |

160 | 161 | 162 |

Parse 163 | options:

164 | 165 | 167 | 168 | 169 | 173 | 174 | 180 |
170 | 171 | 172 |

-c

 175 | 176 | 177 |

Specifies that the log file is 178 | in 'csvlog' format (highly recommended) and not in 'stderr' 179 | format.

181 | 182 |

-b timestamp

183 | 184 |

Only log entries greater or 185 | equal to that timestamp will be parsed. The format is 186 | YYYY-MM-DD HH:MM:SS.FFF like in the log file. An 187 | optional time zone part will be ignored.

188 | 189 |

-e timestamp

190 | 191 |

Only log entries less or equal 192 | to that timestamp will be parsed. The format is 193 | YYYY-MM-DD HH:MM:SS.FFF like in the log file. An 194 | optional time zone part will be ignored.

195 | 196 | 198 | 199 | 200 | 204 | 205 | 213 |
201 | 202 | 203 |

-q

 206 | 207 | 208 |

Specifies that a backslash in a simple string literal 209 | will escape the following single quote. This depends on 210 | configuration options like 211 | standard_conforming_strings and is the default for 212 | server version 9.0 and less.

214 | 215 |

-D database

216 | 217 |

Only log entries related to the 218 | specified database will be parsed (this option can be 219 | specified multiple times for more than one database).

220 | 221 |

-U username

222 | 223 |

Only log entries related to the 224 | specified username will be parsed (this option can be 225 | specified multiple times for more than one user).

226 | 227 |

Replay 228 | options:
229 | -h hostname

230 | 231 |

Host name where the target 232 | database cluster is running (or directory where the UNIX 233 | socket can be found). Defaults to local connections.
234 | This works just like the -h option of 235 | psql.

236 | 237 |

-p port

238 | 239 |

TCP port where the target 240 | database cluster can be reached.

241 | 242 |

-W password

243 | 244 |

By default, pgreplay 245 | assumes that the target database cluster is configured for 246 | trust authentication. With the -W option you 247 | can specify a password that will be used for all users in 248 | the cluster.

249 | 250 |

-s factor

251 | 252 |

Speed factor for replay, by 253 | default 1. This can be any valid positive floating point 254 | number. A factor less than 1 will replay the workload 255 | in ‘slow motion’, while a factor greater 256 | than 1 means ‘fast forward’.

257 | 258 |

-E encoding

259 | 260 |

Specifies the encoding of the 261 | log file, which will be used as client encoding during 262 | replay. If it is omitted, your default client encoding will 263 | be used.

264 | 265 | 267 | 268 | 269 | 273 | 274 | 281 | 282 | 283 | 287 | 288 | 295 |
270 | 271 | 272 |

-j

 275 | 276 | 277 |

If all connections are idle, jump ahead to the next 278 | request instead of sleeping. This will speed up replay. 279 | Execution delays will still be reported correctly, but 280 | replay statistics will not contain the idle time.

284 | 285 | 286 |

-n

 289 | 290 | 291 |

Dry run mode. No connections to the server are made. 292 | Useful for checking if the replay file is corrupt or to get 293 | statistics about the replay file (number of statements, 294 | original duration, ...)

296 | 297 |

-X options

298 | 299 |

Extra connection options for 300 | replay connections. These must be libpq connection options 301 | specified in the format ‘option=value 302 | [...]’.

303 | 304 |

Output 305 | options:
306 | -o outfile

307 | 308 |

specifies the replay file where 309 | the statements will be written for later replay.

310 | 311 |

Debug 312 | options:
313 | -d level

314 | 315 |

Specifies the trace level 316 | (between 1 and 3). Increasing levels will produce more 317 | detailed information about what pgreplay is 318 | doing.

319 | 320 | 322 | 323 | 324 | 328 | 329 | 333 | 335 |
325 | 326 | 327 |

-v

 330 | 331 | 332 |

Prints the program version and exits.

 334 |
336 | 337 |

ENVIRONMENT 338 | 339 |

340 | 341 | 342 | 344 | 345 | 346 | 350 | 351 | 356 | 358 | 359 | 360 | 364 | 365 | 370 | 372 |
347 | 348 | 349 |

PGHOST

 352 | 353 | 354 |

Specifies the default value for 355 | the -h option.

 357 |
361 | 362 | 363 |

PGPORT

 366 | 367 | 368 |

Specifies the default value for the -p 369 | option.

 371 |
373 | 374 |

PGCLIENTENCODING

375 | 376 |

Specifies the default value for 377 | the -E option.

378 | 379 |

LIMITATIONS 380 | 381 |

382 | 383 | 384 | 385 |

pgreplay 386 | can only replay what is logged by PostgreSQL. This leads to 387 | some limitations:

388 | 389 | 391 | 392 | 393 | 397 | 398 | 403 | 404 | 405 | 409 | 410 | 416 | 417 | 418 | 422 | 423 | 430 | 431 | 432 | 436 | 437 | 444 | 445 | 446 | 450 | 451 | 460 | 461 | 462 | 466 | 467 | 475 |
394 | 395 | 396 |

*

 399 | 400 | 401 |

COPY statements will not be replayed, because the 402 | copy data are not logged.

406 | 407 | 408 |

*

 411 | 412 | 413 |

Fast-path API function calls are not logged and will not 414 | be replayed. Unfortunately, this includes the Large Object 415 | API.

419 | 420 | 421 |

*

 424 | 425 | 426 |

Since the log file is always in the server encoding 427 | (which you can specify with the -E switch of 428 | pgreplay), all SET client_encoding statements 429 | will be ignored.

433 | 434 | 435 |

*

 438 | 439 | 440 |

Since the preparation time of prepared statements is not 441 | logged (unless log_min_messages is debug2 or 442 | more), these statements will be prepared immediately before 443 | they are first executed during replay.

447 | 448 | 449 |

*

 452 | 453 | 454 |

Because the log file contains only text, query 455 | parameters and return values will always be in text and 456 | never in binary format. If you use binary mode to, say, 457 | transfer large binary data, pgreplay can cause 458 | significantly more network traffic than the original 459 | run.

463 | 464 | 465 |

*

 468 | 469 | 470 |

Sometimes, if a connection takes longer to complete, the 471 | session ID unexpectedly changes in the PostgreSQL log file. 472 | This causes pgreplay to treat the session as two 473 | different ones, resulting in an additional connection. This 474 | is arguably a bug in PostgreSQL.

476 | 477 |

AUTHOR 478 | 479 |

480 | 481 | 482 |

Written by 483 | Laurenz Albe <laurenz.albe@cybertec.at>.

484 |
485 | 486 | 487 | -------------------------------------------------------------------------------- /install-sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # install - install a program, script, or datafile 3 | 4 | scriptversion=2009-04-28.21; # UTC 5 | 6 | # This originates from X11R5 (mit/util/scripts/install.sh), which was 7 | # later released in X11R6 (xc/config/util/install.sh) with the 8 | # following copyright and license. 9 | # 10 | # Copyright (C) 1994 X Consortium 11 | # 12 | # Permission is hereby granted, free of charge, to any person obtaining a copy 13 | # of this software and associated documentation files (the "Software"), to 14 | # deal in the Software without restriction, including without limitation the 15 | # rights to use, copy, modify, merge, publish, distribute, sublicense, and/or 16 | # sell copies of the Software, and to permit persons to whom the Software is 17 | # furnished to do so, subject to the following conditions: 18 | # 19 | # The above copyright notice and this permission notice shall be included in 20 | # all copies or substantial portions of the Software. 21 | # 22 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 23 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 24 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 25 | # X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN 26 | # AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNEC- 27 | # TION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 28 | # 29 | # Except as contained in this notice, the name of the X Consortium shall not 30 | # be used in advertising or otherwise to promote the sale, use or other deal- 31 | # ings in this Software without prior written authorization from the X Consor- 32 | # tium. 33 | # 34 | # 35 | # FSF changes to this file are in the public domain. 36 | # 37 | # Calling this script install-sh is preferred over install.sh, to prevent 38 | # `make' implicit rules from creating a file called install from it 39 | # when there is no Makefile. 40 | # 41 | # This script is compatible with the BSD install script, but was written 42 | # from scratch. 43 | 44 | nl=' 45 | ' 46 | IFS=" "" $nl" 47 | 48 | # set DOITPROG to echo to test this script 49 | 50 | # Don't use :- since 4.3BSD and earlier shells don't like it. 51 | doit=${DOITPROG-} 52 | if test -z "$doit"; then 53 | doit_exec=exec 54 | else 55 | doit_exec=$doit 56 | fi 57 | 58 | # Put in absolute file names if you don't have them in your path; 59 | # or use environment vars. 60 | 61 | chgrpprog=${CHGRPPROG-chgrp} 62 | chmodprog=${CHMODPROG-chmod} 63 | chownprog=${CHOWNPROG-chown} 64 | cmpprog=${CMPPROG-cmp} 65 | cpprog=${CPPROG-cp} 66 | mkdirprog=${MKDIRPROG-mkdir} 67 | mvprog=${MVPROG-mv} 68 | rmprog=${RMPROG-rm} 69 | stripprog=${STRIPPROG-strip} 70 | 71 | posix_glob='?' 72 | initialize_posix_glob=' 73 | test "$posix_glob" != "?" || { 74 | if (set -f) 2>/dev/null; then 75 | posix_glob= 76 | else 77 | posix_glob=: 78 | fi 79 | } 80 | ' 81 | 82 | posix_mkdir= 83 | 84 | # Desired mode of installed file. 85 | mode=0755 86 | 87 | chgrpcmd= 88 | chmodcmd=$chmodprog 89 | chowncmd= 90 | mvcmd=$mvprog 91 | rmcmd="$rmprog -f" 92 | stripcmd= 93 | 94 | src= 95 | dst= 96 | dir_arg= 97 | dst_arg= 98 | 99 | copy_on_change=false 100 | no_target_directory= 101 | 102 | usage="\ 103 | Usage: $0 [OPTION]... [-T] SRCFILE DSTFILE 104 | or: $0 [OPTION]... SRCFILES... DIRECTORY 105 | or: $0 [OPTION]... -t DIRECTORY SRCFILES... 106 | or: $0 [OPTION]... -d DIRECTORIES... 107 | 108 | In the 1st form, copy SRCFILE to DSTFILE. 109 | In the 2nd and 3rd, copy all SRCFILES to DIRECTORY. 110 | In the 4th, create DIRECTORIES. 111 | 112 | Options: 113 | --help display this help and exit. 114 | --version display version info and exit. 115 | 116 | -c (ignored) 117 | -C install only if different (preserve the last data modification time) 118 | -d create directories instead of installing files. 119 | -g GROUP $chgrpprog installed files to GROUP. 120 | -m MODE $chmodprog installed files to MODE. 121 | -o USER $chownprog installed files to USER. 122 | -s $stripprog installed files. 123 | -t DIRECTORY install into DIRECTORY. 124 | -T report an error if DSTFILE is a directory. 125 | 126 | Environment variables override the default commands: 127 | CHGRPPROG CHMODPROG CHOWNPROG CMPPROG CPPROG MKDIRPROG MVPROG 128 | RMPROG STRIPPROG 129 | " 130 | 131 | while test $# -ne 0; do 132 | case $1 in 133 | -c) ;; 134 | 135 | -C) copy_on_change=true;; 136 | 137 | -d) dir_arg=true;; 138 | 139 | -g) chgrpcmd="$chgrpprog $2" 140 | shift;; 141 | 142 | --help) echo "$usage"; exit $?;; 143 | 144 | -m) mode=$2 145 | case $mode in 146 | *' '* | *' '* | *' 147 | '* | *'*'* | *'?'* | *'['*) 148 | echo "$0: invalid mode: $mode" >&2 149 | exit 1;; 150 | esac 151 | shift;; 152 | 153 | -o) chowncmd="$chownprog $2" 154 | shift;; 155 | 156 | -s) stripcmd=$stripprog;; 157 | 158 | -t) dst_arg=$2 159 | shift;; 160 | 161 | -T) no_target_directory=true;; 162 | 163 | --version) echo "$0 $scriptversion"; exit $?;; 164 | 165 | --) shift 166 | break;; 167 | 168 | -*) echo "$0: invalid option: $1" >&2 169 | exit 1;; 170 | 171 | *) break;; 172 | esac 173 | shift 174 | done 175 | 176 | if test $# -ne 0 && test -z "$dir_arg$dst_arg"; then 177 | # When -d is used, all remaining arguments are directories to create. 178 | # When -t is used, the destination is already specified. 179 | # Otherwise, the last argument is the destination. Remove it from $@. 180 | for arg 181 | do 182 | if test -n "$dst_arg"; then 183 | # $@ is not empty: it contains at least $arg. 184 | set fnord "$@" "$dst_arg" 185 | shift # fnord 186 | fi 187 | shift # arg 188 | dst_arg=$arg 189 | done 190 | fi 191 | 192 | if test $# -eq 0; then 193 | if test -z "$dir_arg"; then 194 | echo "$0: no input file specified." >&2 195 | exit 1 196 | fi 197 | # It's OK to call `install-sh -d' without argument. 198 | # This can happen when creating conditional directories. 199 | exit 0 200 | fi 201 | 202 | if test -z "$dir_arg"; then 203 | trap '(exit $?); exit' 1 2 13 15 204 | 205 | # Set umask so as not to create temps with too-generous modes. 206 | # However, 'strip' requires both read and write access to temps. 207 | case $mode in 208 | # Optimize common cases. 209 | *644) cp_umask=133;; 210 | *755) cp_umask=22;; 211 | 212 | *[0-7]) 213 | if test -z "$stripcmd"; then 214 | u_plus_rw= 215 | else 216 | u_plus_rw='% 200' 217 | fi 218 | cp_umask=`expr '(' 777 - $mode % 1000 ')' $u_plus_rw`;; 219 | *) 220 | if test -z "$stripcmd"; then 221 | u_plus_rw= 222 | else 223 | u_plus_rw=,u+rw 224 | fi 225 | cp_umask=$mode$u_plus_rw;; 226 | esac 227 | fi 228 | 229 | for src 230 | do 231 | # Protect names starting with `-'. 232 | case $src in 233 | -*) src=./$src;; 234 | esac 235 | 236 | if test -n "$dir_arg"; then 237 | dst=$src 238 | dstdir=$dst 239 | test -d "$dstdir" 240 | dstdir_status=$? 241 | else 242 | 243 | # Waiting for this to be detected by the "$cpprog $src $dsttmp" command 244 | # might cause directories to be created, which would be especially bad 245 | # if $src (and thus $dsttmp) contains '*'. 246 | if test ! -f "$src" && test ! -d "$src"; then 247 | echo "$0: $src does not exist." >&2 248 | exit 1 249 | fi 250 | 251 | if test -z "$dst_arg"; then 252 | echo "$0: no destination specified." >&2 253 | exit 1 254 | fi 255 | 256 | dst=$dst_arg 257 | # Protect names starting with `-'. 258 | case $dst in 259 | -*) dst=./$dst;; 260 | esac 261 | 262 | # If destination is a directory, append the input filename; won't work 263 | # if double slashes aren't ignored. 264 | if test -d "$dst"; then 265 | if test -n "$no_target_directory"; then 266 | echo "$0: $dst_arg: Is a directory" >&2 267 | exit 1 268 | fi 269 | dstdir=$dst 270 | dst=$dstdir/`basename "$src"` 271 | dstdir_status=0 272 | else 273 | # Prefer dirname, but fall back on a substitute if dirname fails. 274 | dstdir=` 275 | (dirname "$dst") 2>/dev/null || 276 | expr X"$dst" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ 277 | X"$dst" : 'X\(//\)[^/]' \| \ 278 | X"$dst" : 'X\(//\)$' \| \ 279 | X"$dst" : 'X\(/\)' \| . 2>/dev/null || 280 | echo X"$dst" | 281 | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ 282 | s//\1/ 283 | q 284 | } 285 | /^X\(\/\/\)[^/].*/{ 286 | s//\1/ 287 | q 288 | } 289 | /^X\(\/\/\)$/{ 290 | s//\1/ 291 | q 292 | } 293 | /^X\(\/\).*/{ 294 | s//\1/ 295 | q 296 | } 297 | s/.*/./; q' 298 | ` 299 | 300 | test -d "$dstdir" 301 | dstdir_status=$? 302 | fi 303 | fi 304 | 305 | obsolete_mkdir_used=false 306 | 307 | if test $dstdir_status != 0; then 308 | case $posix_mkdir in 309 | '') 310 | # Create intermediate dirs using mode 755 as modified by the umask. 311 | # This is like FreeBSD 'install' as of 1997-10-28. 312 | umask=`umask` 313 | case $stripcmd.$umask in 314 | # Optimize common cases. 315 | *[2367][2367]) mkdir_umask=$umask;; 316 | .*0[02][02] | .[02][02] | .[02]) mkdir_umask=22;; 317 | 318 | *[0-7]) 319 | mkdir_umask=`expr $umask + 22 \ 320 | - $umask % 100 % 40 + $umask % 20 \ 321 | - $umask % 10 % 4 + $umask % 2 322 | `;; 323 | *) mkdir_umask=$umask,go-w;; 324 | esac 325 | 326 | # With -d, create the new directory with the user-specified mode. 327 | # Otherwise, rely on $mkdir_umask. 328 | if test -n "$dir_arg"; then 329 | mkdir_mode=-m$mode 330 | else 331 | mkdir_mode= 332 | fi 333 | 334 | posix_mkdir=false 335 | case $umask in 336 | *[123567][0-7][0-7]) 337 | # POSIX mkdir -p sets u+wx bits regardless of umask, which 338 | # is incompatible with FreeBSD 'install' when (umask & 300) != 0. 339 | ;; 340 | *) 341 | tmpdir=${TMPDIR-/tmp}/ins$RANDOM-$$ 342 | trap 'ret=$?; rmdir "$tmpdir/d" "$tmpdir" 2>/dev/null; exit $ret' 0 343 | 344 | if (umask $mkdir_umask && 345 | exec $mkdirprog $mkdir_mode -p -- "$tmpdir/d") >/dev/null 2>&1 346 | then 347 | if test -z "$dir_arg" || { 348 | # Check for POSIX incompatibilities with -m. 349 | # HP-UX 11.23 and IRIX 6.5 mkdir -m -p sets group- or 350 | # other-writeable bit of parent directory when it shouldn't. 351 | # FreeBSD 6.1 mkdir -m -p sets mode of existing directory. 352 | ls_ld_tmpdir=`ls -ld "$tmpdir"` 353 | case $ls_ld_tmpdir in 354 | d????-?r-*) different_mode=700;; 355 | d????-?--*) different_mode=755;; 356 | *) false;; 357 | esac && 358 | $mkdirprog -m$different_mode -p -- "$tmpdir" && { 359 | ls_ld_tmpdir_1=`ls -ld "$tmpdir"` 360 | test "$ls_ld_tmpdir" = "$ls_ld_tmpdir_1" 361 | } 362 | } 363 | then posix_mkdir=: 364 | fi 365 | rmdir "$tmpdir/d" "$tmpdir" 366 | else 367 | # Remove any dirs left behind by ancient mkdir implementations. 368 | rmdir ./$mkdir_mode ./-p ./-- 2>/dev/null 369 | fi 370 | trap '' 0;; 371 | esac;; 372 | esac 373 | 374 | if 375 | $posix_mkdir && ( 376 | umask $mkdir_umask && 377 | $doit_exec $mkdirprog $mkdir_mode -p -- "$dstdir" 378 | ) 379 | then : 380 | else 381 | 382 | # The umask is ridiculous, or mkdir does not conform to POSIX, 383 | # or it failed possibly due to a race condition. Create the 384 | # directory the slow way, step by step, checking for races as we go. 385 | 386 | case $dstdir in 387 | /*) prefix='/';; 388 | -*) prefix='./';; 389 | *) prefix='';; 390 | esac 391 | 392 | eval "$initialize_posix_glob" 393 | 394 | oIFS=$IFS 395 | IFS=/ 396 | $posix_glob set -f 397 | set fnord $dstdir 398 | shift 399 | $posix_glob set +f 400 | IFS=$oIFS 401 | 402 | prefixes= 403 | 404 | for d 405 | do 406 | test -z "$d" && continue 407 | 408 | prefix=$prefix$d 409 | if test -d "$prefix"; then 410 | prefixes= 411 | else 412 | if $posix_mkdir; then 413 | (umask=$mkdir_umask && 414 | $doit_exec $mkdirprog $mkdir_mode -p -- "$dstdir") && break 415 | # Don't fail if two instances are running concurrently. 416 | test -d "$prefix" || exit 1 417 | else 418 | case $prefix in 419 | *\'*) qprefix=`echo "$prefix" | sed "s/'/'\\\\\\\\''/g"`;; 420 | *) qprefix=$prefix;; 421 | esac 422 | prefixes="$prefixes '$qprefix'" 423 | fi 424 | fi 425 | prefix=$prefix/ 426 | done 427 | 428 | if test -n "$prefixes"; then 429 | # Don't fail if two instances are running concurrently. 430 | (umask $mkdir_umask && 431 | eval "\$doit_exec \$mkdirprog $prefixes") || 432 | test -d "$dstdir" || exit 1 433 | obsolete_mkdir_used=true 434 | fi 435 | fi 436 | fi 437 | 438 | if test -n "$dir_arg"; then 439 | { test -z "$chowncmd" || $doit $chowncmd "$dst"; } && 440 | { test -z "$chgrpcmd" || $doit $chgrpcmd "$dst"; } && 441 | { test "$obsolete_mkdir_used$chowncmd$chgrpcmd" = false || 442 | test -z "$chmodcmd" || $doit $chmodcmd $mode "$dst"; } || exit 1 443 | else 444 | 445 | # Make a couple of temp file names in the proper directory. 446 | dsttmp=$dstdir/_inst.$$_ 447 | rmtmp=$dstdir/_rm.$$_ 448 | 449 | # Trap to clean up those temp files at exit. 450 | trap 'ret=$?; rm -f "$dsttmp" "$rmtmp" && exit $ret' 0 451 | 452 | # Copy the file name to the temp name. 453 | (umask $cp_umask && $doit_exec $cpprog "$src" "$dsttmp") && 454 | 455 | # and set any options; do chmod last to preserve setuid bits. 456 | # 457 | # If any of these fail, we abort the whole thing. If we want to 458 | # ignore errors from any of these, just make sure not to ignore 459 | # errors from the above "$doit $cpprog $src $dsttmp" command. 460 | # 461 | { test -z "$chowncmd" || $doit $chowncmd "$dsttmp"; } && 462 | { test -z "$chgrpcmd" || $doit $chgrpcmd "$dsttmp"; } && 463 | { test -z "$stripcmd" || $doit $stripcmd "$dsttmp"; } && 464 | { test -z "$chmodcmd" || $doit $chmodcmd $mode "$dsttmp"; } && 465 | 466 | # If -C, don't bother to copy if it wouldn't change the file. 467 | if $copy_on_change && 468 | old=`LC_ALL=C ls -dlL "$dst" 2>/dev/null` && 469 | new=`LC_ALL=C ls -dlL "$dsttmp" 2>/dev/null` && 470 | 471 | eval "$initialize_posix_glob" && 472 | $posix_glob set -f && 473 | set X $old && old=:$2:$4:$5:$6 && 474 | set X $new && new=:$2:$4:$5:$6 && 475 | $posix_glob set +f && 476 | 477 | test "$old" = "$new" && 478 | $cmpprog "$dst" "$dsttmp" >/dev/null 2>&1 479 | then 480 | rm -f "$dsttmp" 481 | else 482 | # Rename the file to the real destination. 483 | $doit $mvcmd -f "$dsttmp" "$dst" 2>/dev/null || 484 | 485 | # The rename failed, perhaps because mv can't rename something else 486 | # to itself, or perhaps because mv is so ancient that it does not 487 | # support -f. 488 | { 489 | # Now remove or move aside any old file at destination location. 490 | # We try this two ways since rm can't unlink itself on some 491 | # systems and the destination file might be busy for other 492 | # reasons. In this case, the final cleanup might fail but the new 493 | # file should still install successfully. 494 | { 495 | test ! -f "$dst" || 496 | $doit $rmcmd -f "$dst" 2>/dev/null || 497 | { $doit $mvcmd -f "$dst" "$rmtmp" 2>/dev/null && 498 | { $doit $rmcmd -f "$rmtmp" 2>/dev/null; :; } 499 | } || 500 | { echo "$0: cannot unlink or rename $dst" >&2 501 | (exit 1); exit 1 502 | } 503 | } && 504 | 505 | # Now rename the file to the real destination. 506 | $doit $mvcmd "$dsttmp" "$dst" 507 | } 508 | fi || exit 1 509 | 510 | trap '' 0 511 | fi 512 | done 513 | 514 | # Local variables: 515 | # eval: (add-hook 'write-file-hooks 'time-stamp) 516 | # time-stamp-start: "scriptversion=" 517 | # time-stamp-format: "%:y-%02m-%02d.%02H" 518 | # time-stamp-time-zone: "UTC" 519 | # time-stamp-end: "; # UTC" 520 | # End: 521 | -------------------------------------------------------------------------------- /database.c: -------------------------------------------------------------------------------- 1 | #include "pgreplay.h" 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #ifdef HAVE_SYS_SELECT_H 9 | # include 10 | #else 11 | # include 12 | # include 13 | #endif 14 | #ifdef TIME_WITH_SYS_TIME 15 | # include 16 | # include 17 | #else 18 | # ifdef HAVE_SYS_TIME_H 19 | # include 20 | # else 21 | # include 22 | # endif 23 | #endif 24 | #ifdef WINDOWS 25 | # include 26 | #endif 27 | 28 | /* 29 | * Utility macros to calculate with struct timeval. 30 | * These are already defined on BSD type systems. 31 | */ 32 | 33 | #ifndef timeradd 34 | # define timeradd(a, b, result) \ 35 | do { \ 36 | (result)->tv_sec = (a)->tv_sec + (b)->tv_sec; \ 37 | (result)->tv_usec = (a)->tv_usec + (b)->tv_usec; \ 38 | if ((result)->tv_usec >= 1000000) { \ 39 | ++(result)->tv_sec; \ 40 | (result)->tv_usec -= 1000000; \ 41 | } \ 42 | } while (0) 43 | #endif 44 | 45 | #ifndef timersub 46 | # define timersub(a, b, result) \ 47 | do { \ 48 | (result)->tv_sec = (a)->tv_sec - (b)->tv_sec; \ 49 | (result)->tv_usec = (a)->tv_usec - (b)->tv_usec; \ 50 | if ((result)->tv_usec < 0) { \ 51 | --(result)->tv_sec; \ 52 | (result)->tv_usec += 1000000; \ 53 | } \ 54 | } while (0) 55 | #endif 56 | 57 | /* connect string */ 58 | static char *conn_string; 59 | 60 | /* speed factor for replay */ 61 | static double replay_factor; 62 | 63 | /* possible stati a connection can have */ 64 | typedef enum { 65 | idle = 0, 66 | conn_wait_write, 67 | conn_wait_read, 68 | wait_write, 69 | wait_read, 70 | closed 71 | } connstatus; 72 | 73 | /* linked list element for list of open connections */ 74 | struct dbconn { 75 | uint64_t session_id; 76 | PGconn *db_conn; 77 | int socket; 78 | connstatus status; 79 | struct timeval session_start; 80 | struct timeval stmt_start; 81 | char *errmsg; 82 | struct dbconn *next; 83 | }; 84 | 85 | /* linked list of open connections */ 86 | static struct dbconn *connections = NULL; 87 | 88 | /* remember timestamp of program start and stop */ 89 | static struct timeval start_time; 90 | static struct timeval stop_time; 91 | 92 | /* remember timestamp of first statement */ 93 | static struct timeval first_stmt_time; 94 | static struct timeval last_stmt_time; 95 | 96 | /* maximum seconds behind schedule */ 97 | static time_t secs_behind = 0; 98 | 99 | /* time skipped instead of sleeping through it */ 100 | static struct timeval jump_total = {0, 0}; 101 | 102 | /* statistics */ 103 | static struct timeval stat_exec = {0, 0}; /* SQL statement execution time */ 104 | static struct timeval stat_session = {0, 0}; /* session duration total */ 105 | static struct timeval stat_longstmt = {0, 0}; /* session duration total */ 106 | static unsigned long stat_stmt = 0; /* number of SQL statements */ 107 | static unsigned long stat_prep = 0; /* number of preparations */ 108 | static unsigned long stat_errors = 0; /* unsuccessful SQL statements and preparations */ 109 | static unsigned long stat_actions = 0; /* client-server interactions */ 110 | static unsigned long stat_statements = 0; /* number of concurrent statements */ 111 | static unsigned long stat_stmtmax = 0; /* maximum concurrent statements */ 112 | static unsigned long stat_sesscnt = 0; /* total number of sessions */ 113 | static unsigned long stat_sessions = 0; /* number of concurrent sessions */ 114 | static unsigned long stat_sessmax = 0; /* maximum concurrent sessions */ 115 | static unsigned long stat_hist[5] = {0, 0, 0, 0, 0}; /* duration histogram */ 116 | 117 | #define NUM_DELAY_STEPS 11 118 | 119 | /* steps for execution delay reports */ 120 | static struct { 121 | int seconds; 122 | char *display; 123 | short int shown; 124 | } delay_steps[NUM_DELAY_STEPS] = { 125 | {10, "10 seconds", 0}, 126 | {30, "30 seconds", 0}, 127 | {60, "1 minute", 0}, 128 | {180, "3 minutes", 0}, 129 | {600, "10 minutes", 0}, 130 | {1800, "30 minutes", 0}, 131 | {3600, "1 hour", 0}, 132 | {7200, "2 hours", 0}, 133 | {21600, "6 hours", 0}, 134 | {43200, "12 hours", 0}, 135 | {86400, "1 day", 0} 136 | }; 137 | 138 | /* processes (ignores) notices from the server */ 139 | static void ignore_notices(void *arg, const PGresult *res) { 140 | } 141 | 142 | /* encapsulates "select" call and error handling */ 143 | 144 | static int do_select(int n, fd_set *rfds, fd_set *wfds, fd_set *xfds, struct timeval *timeout) { 145 | int rc; 146 | 147 | rc = select(n, rfds, wfds, xfds, timeout); 148 | #ifdef WINDOWS 149 | if (SOCKET_ERROR == rc) { 150 | win_perror("Error in select()", 1); 151 | rc = -1; 152 | } 153 | #else 154 | if (-1 == rc) { 155 | perror("Error in select()"); 156 | } 157 | #endif 158 | 159 | return rc; 160 | } 161 | 162 | /* checks if a certain socket can be read or written without blocking */ 163 | 164 | static int poll_socket(int socket, int for_read, char * const errmsg_prefix) { 165 | fd_set fds; 166 | struct timeval zero = { 0, 0 }; 167 | 168 | FD_ZERO(&fds); 169 | FD_SET(socket, &fds); 170 | return do_select(socket + 1, for_read ? &fds : NULL, for_read ? NULL : &fds, NULL, &zero); 171 | } 172 | 173 | /* sleep routine that should work on all platforms */ 174 | 175 | static int do_sleep(struct timeval *delta) { 176 | debug(2, "Napping for %lu.%06lu seconds\n", (unsigned long)delta->tv_sec, (unsigned long)delta->tv_usec); 177 | #ifdef WINDOWS 178 | Sleep((DWORD)delta->tv_sec * 1000 + (DWORD)(delta->tv_usec / 1000)); 179 | return 0; 180 | #else 181 | return do_select(0, NULL, NULL, NULL, delta); 182 | #endif 183 | } 184 | 185 | static void print_replay_statistics(int dry_run) { 186 | int hours, minutes; 187 | double seconds, runtime, session_time, busy_time; 188 | struct timeval delta; 189 | unsigned long histtotal = 190 | stat_hist[0] + stat_hist[1] + stat_hist[2] + stat_hist[3] + stat_hist[4]; 191 | 192 | if (dry_run) { 193 | fprintf(sf, "\nReplay statistics (dry run)\n"); 194 | fprintf(sf, "===========================\n\n"); 195 | 196 | /* calculate lengh of the recorded workload */ 197 | timersub(&last_stmt_time, &first_stmt_time, &delta); 198 | hours = delta.tv_sec / 3600; 199 | delta.tv_sec -= hours * 3600; 200 | minutes = delta.tv_sec / 60; 201 | delta.tv_sec -= minutes * 60; 202 | seconds = delta.tv_usec / 1000000.0 + delta.tv_sec; 203 | 204 | fprintf(sf, "Duration of recorded workload:"); 205 | if (hours > 0) { 206 | fprintf(sf, " %d hours", hours); 207 | } 208 | if (minutes > 0) { 209 | fprintf(sf, " %d minutes", minutes); 210 | } 211 | fprintf(sf, " %.3f seconds\n", seconds); 212 | fprintf(sf, "Calls to the server: %lu\n", stat_actions); 213 | } else { 214 | fprintf(sf, "\nReplay statistics\n"); 215 | fprintf(sf, "=================\n\n"); 216 | 217 | /* calculate total run time */ 218 | timersub(&stop_time, &start_time, &delta); 219 | runtime = delta.tv_usec / 1000000.0 + delta.tv_sec; 220 | /* calculate hours and minutes, subtract from delta */ 221 | hours = delta.tv_sec / 3600; 222 | delta.tv_sec -= hours * 3600; 223 | minutes = delta.tv_sec / 60; 224 | delta.tv_sec -= minutes * 60; 225 | seconds = delta.tv_usec / 1000000.0 + delta.tv_sec; 226 | /* calculate total busy time */ 227 | busy_time = stat_exec.tv_usec / 1000000.0 + stat_exec.tv_sec; 228 | /* calculate total session time */ 229 | session_time = stat_session.tv_usec / 1000000.0 + stat_session.tv_sec; 230 | 231 | fprintf(sf, "Speed factor for replay: %.3f\n", replay_factor); 232 | fprintf(sf, "Total run time:"); 233 | if (hours > 0) { 234 | fprintf(sf, " %d hours", hours); 235 | } 236 | if (minutes > 0) { 237 | fprintf(sf, " %d minutes", minutes); 238 | } 239 | fprintf(sf, " %.3f seconds\n", seconds); 240 | fprintf(sf, "Maximum lag behind schedule: %lu seconds\n", (unsigned long) secs_behind); 241 | fprintf(sf, "Calls to the server: %lu\n", stat_actions); 242 | if (runtime > 0.0) { 243 | fprintf(sf, "(%.3f calls per second)\n", stat_actions / runtime); 244 | } 245 | } 246 | 247 | fprintf(sf, "Total number of connections: %lu\n", stat_sesscnt); 248 | fprintf(sf, "Maximum number of concurrent connections: %lu\n", stat_sessmax); 249 | if (!dry_run && runtime > 0.0) { 250 | fprintf(sf, "Average number of concurrent connections: %.3f\n", session_time / runtime); 251 | } 252 | if (!dry_run && session_time > 0.0) { 253 | fprintf(sf, "Average session idle percentage: %.3f%%\n", 100.0 * (session_time - busy_time) / session_time); 254 | } 255 | 256 | fprintf(sf, "SQL statements executed: %lu\n", stat_stmt - stat_prep); 257 | if (!dry_run && stat_stmt > stat_prep) { 258 | fprintf(sf, "(%lu or %.3f%% of these completed with error)\n", 259 | stat_errors, (100.0 * stat_errors) / (stat_stmt - stat_prep)); 260 | fprintf(sf, "Maximum number of concurrent SQL statements: %lu\n", stat_stmtmax); 261 | if (runtime > 0.0) { 262 | fprintf(sf, "Average number of concurrent SQL statements: %.3f\n", busy_time / runtime); 263 | } 264 | fprintf(sf, "Average SQL statement duration: %.3f seconds\n", busy_time / stat_stmt); 265 | fprintf(sf, "Maximum SQL statement duration: %.3f seconds\n", 266 | stat_longstmt.tv_sec + stat_longstmt.tv_usec / 1000000.0); 267 | fprintf(sf, "Statement duration histogram:\n"); 268 | fprintf(sf, " 0 to 0.02 seconds: %.3f%%\n", 100.0 * stat_hist[0] / histtotal); 269 | fprintf(sf, " 0.02 to 0.1 seconds: %.3f%%\n", 100.0 * stat_hist[1] / histtotal); 270 | fprintf(sf, " 0.1 to 0.5 seconds: %.3f%%\n", 100.0 * stat_hist[2] / histtotal); 271 | fprintf(sf, " 0.5 to 2 seconds: %.3f%%\n", 100.0 * stat_hist[3] / histtotal); 272 | fprintf(sf, " over 2 seconds: %.3f%%\n", 100.0 * stat_hist[4] / histtotal); 273 | } 274 | } 275 | 276 | int database_consumer_init(const char *ignore, const char *host, int port, const char *passwd, double factor) { 277 | int conn_string_len = 12; /* port and '\0' */ 278 | const char *p; 279 | char *p1; 280 | 281 | debug(3, "Entering database_consumer_init%s\n", ""); 282 | 283 | /* get time of program start */ 284 | if (-1 == gettimeofday(&start_time, NULL)) { 285 | perror("Error calling gettimeofday"); 286 | return 0; 287 | } 288 | 289 | replay_factor = factor; 290 | 291 | /* calculate length of connect string */ 292 | if (host) { 293 | conn_string_len += 8; 294 | for (p=host; '\0'!=*p; ++p) { 295 | if (('\'' == *p) || ('\\' == *p)) { 296 | conn_string_len += 2; 297 | } else { 298 | ++conn_string_len; 299 | } 300 | } 301 | } 302 | if (passwd) { 303 | conn_string_len += 12; 304 | for (p=passwd; '\0'!=*p; ++p) { 305 | if (('\'' == *p) || ('\\' == *p)) { 306 | conn_string_len += 2; 307 | } else { 308 | ++conn_string_len; 309 | } 310 | } 311 | } 312 | 313 | if (extra_connstr) 314 | conn_string_len += strlen(extra_connstr); 315 | 316 | if (NULL == (conn_string = malloc(conn_string_len))) { 317 | fprintf(stderr, "Cannot allocate %d bytes of memory\n", conn_string_len); 318 | return 0; 319 | } 320 | /* write the port to the connection string if it is set */ 321 | if (-1 == port) { 322 | conn_string[0] = '\0'; 323 | } else { 324 | if (sprintf(conn_string, "port=%d", port) < 0) { 325 | perror("Error writing connect string:"); 326 | free(conn_string); 327 | return 0; 328 | } 329 | } 330 | for (p1=conn_string; '\0'!=*p1; ++p1) { 331 | /* places p1 at the end of the string */ 332 | } 333 | 334 | /* append host if necessary */ 335 | if (host) { 336 | *(p1++) = ' '; 337 | *(p1++) = 'h'; 338 | *(p1++) = 'o'; 339 | *(p1++) = 's'; 340 | *(p1++) = 't'; 341 | *(p1++) = '='; 342 | *(p1++) = '\''; 343 | for (p=host; '\0'!=*p; ++p) { 344 | if (('\'' == *p) || ('\\' == *p)) { 345 | *(p1++) = '\\'; 346 | } 347 | *(p1++) = *p; 348 | } 349 | *(p1++) = '\''; 350 | *p1 = '\0'; 351 | } 352 | 353 | /* append password if necessary */ 354 | if (passwd) { 355 | *(p1++) = ' '; 356 | *(p1++) = 'p'; 357 | *(p1++) = 'a'; 358 | *(p1++) = 's'; 359 | *(p1++) = 's'; 360 | *(p1++) = 'w'; 361 | *(p1++) = 'o'; 362 | *(p1++) = 'r'; 363 | *(p1++) = 'd'; 364 | *(p1++) = '='; 365 | *(p1++) = '\''; 366 | for (p=passwd; '\0'!=*p; ++p) { 367 | if (('\'' == *p) || ('\\' == *p)) { 368 | *(p1++) = '\\'; 369 | } 370 | *(p1++) = *p; 371 | } 372 | *(p1++) = '\''; 373 | *p1 = '\0'; 374 | } 375 | 376 | if (extra_connstr) { 377 | *(p1++) = ' '; 378 | strcpy(p1, extra_connstr); 379 | } 380 | 381 | debug(2, "Database connect string: \"%s\"\n", conn_string); 382 | 383 | debug(3, "Leaving database_consumer_init%s\n", ""); 384 | return 1; 385 | } 386 | 387 | void database_consumer_finish(int dry_run) { 388 | debug(3, "Entering database_consumer_finish%s\n", ""); 389 | 390 | free(conn_string); 391 | 392 | if (NULL != connections) { 393 | fprintf(stderr, "Error: not all database connections closed\n"); 394 | } 395 | 396 | if (-1 == gettimeofday(&stop_time, NULL)) { 397 | perror("Error calling gettimeofday"); 398 | } else if (sf) { 399 | print_replay_statistics(dry_run); 400 | } 401 | 402 | debug(3, "Leaving database_consumer_finish%s\n", ""); 403 | } 404 | 405 | int database_consumer(replay_item *item) { 406 | const uint64_t session_id = replay_get_session_id(item); 407 | const replay_type type = replay_get_type(item); 408 | int all_idle = 1, rc = 0, j; 409 | struct dbconn *conn = connections, *found_conn = NULL, *prev_conn = NULL; 410 | struct timeval target_time, now, delta; 411 | const struct timeval *stmt_time; 412 | static int fstmtm_set = 0; /* have we already collected first_statement_time */ 413 | double d; 414 | time_t i; 415 | char *connstr, *p1, errbuf[256]; 416 | const char *user, *database, *p; 417 | PGcancel *cancel_request; 418 | PGresult *result; 419 | ExecStatusType result_status; 420 | 421 | debug(3, "Entering database_consumer%s\n", ""); 422 | 423 | /* loop through open connections and do what can be done */ 424 | while ((-1 != rc) && (NULL != conn)) { 425 | /* if we find the connection for the current statement, remember it */ 426 | if (session_id == conn->session_id) { 427 | found_conn = conn; 428 | } 429 | 430 | /* handle each connection according to status */ 431 | switch(conn->status) { 432 | case idle: 433 | case closed: 434 | break; /* nothing to do */ 435 | 436 | case conn_wait_read: 437 | case conn_wait_write: 438 | /* in connection process */ 439 | /* check if socket is still busy */ 440 | switch (poll_socket(conn->socket, (conn_wait_read == conn->status), "Error polling socket during connect")) { 441 | case 0: 442 | /* socket still busy */ 443 | debug(2, "Socket for session 0x" UINT64_FORMAT " busy for %s during connect\n", conn->session_id, (conn_wait_write == conn->status) ? "write" : "read"); 444 | all_idle = 0; 445 | break; 446 | case -1: 447 | /* error happened in select() */ 448 | rc = -1; 449 | default: 450 | /* socket not busy, continue connect process */ 451 | switch(PQconnectPoll(conn->db_conn)) { 452 | case PGRES_POLLING_WRITING: 453 | conn->status = conn_wait_write; 454 | all_idle = 0; 455 | break; 456 | case PGRES_POLLING_READING: 457 | conn->status = conn_wait_read; 458 | all_idle = 0; 459 | break; 460 | case PGRES_POLLING_OK: 461 | debug(2, "Connection for session 0x" UINT64_FORMAT " established\n", conn->session_id); 462 | conn->status = idle; 463 | 464 | /* get session start time */ 465 | if (-1 == gettimeofday(&(conn->session_start), NULL)) { 466 | perror("Error calling gettimeofday"); 467 | rc = -1; 468 | } 469 | 470 | /* count total and concurrent sessions */ 471 | ++stat_sesscnt; 472 | if (++stat_sessions > stat_sessmax) { 473 | stat_sessmax = stat_sessions; 474 | } 475 | 476 | break; 477 | case PGRES_POLLING_FAILED: 478 | /* If the connection fails because of a 479 | FATAL error from the server, mark 480 | connection "closed" and keep going. 481 | The same thing probably happened in the 482 | original run. 483 | PostgreSQL logs no disconnection for this. 484 | */ 485 | p1 = PQerrorMessage(conn->db_conn); 486 | if (0 == strncmp(p1, "FATAL: ", 8)) { 487 | p1 += 8; 488 | if (NULL == (conn->errmsg = malloc(strlen(p1) + 1))) { 489 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)(strlen(p1) + 1)); 490 | rc = -1; 491 | } else { 492 | debug(2, "Connection for session 0x" UINT64_FORMAT " failed with FATAL error: %s\n", 493 | conn->session_id, p1); 494 | strcpy(conn->errmsg, p1); 495 | conn->status = closed; 496 | PQfinish(conn->db_conn); 497 | } 498 | 499 | break; 500 | } 501 | /* else fall through */ 502 | default: 503 | fprintf(stderr, "Connection for session 0x" UINT64_FORMAT " failed: %s\n", conn->session_id, PQerrorMessage(conn->db_conn)); 504 | rc = -1; 505 | PQfinish(conn->db_conn); 506 | } 507 | break; 508 | } 509 | break; 510 | 511 | case wait_write: 512 | /* check if the socket is writable */ 513 | switch (poll_socket(conn->socket, 0, "Error polling socket for write")) { 514 | case 0: 515 | /* socket still busy */ 516 | debug(2, "Session 0x" UINT64_FORMAT " busy writing data\n", conn->session_id); 517 | all_idle = 0; 518 | break; 519 | case -1: 520 | /* error in select() */ 521 | rc = -1; 522 | default: 523 | /* try PQflush again */ 524 | debug(2, "Session 0x" UINT64_FORMAT " flushing data\n", conn->session_id); 525 | switch (PQflush(conn->db_conn)) { 526 | case 0: 527 | /* finished flushing all data */ 528 | conn->status = wait_read; 529 | all_idle = 0; 530 | break; 531 | case 1: 532 | /* more data to flush */ 533 | all_idle = 0; 534 | break; 535 | default: 536 | fprintf(stderr, "Error flushing to database: %s\n", PQerrorMessage(conn->db_conn)); 537 | rc = -1; 538 | } 539 | break; 540 | } 541 | break; 542 | 543 | case wait_read: 544 | /* check if the socket is readable */ 545 | switch (poll_socket(conn->socket, 1, "Error polling socket for read")) { 546 | case 0: 547 | /* socket still busy */ 548 | debug(2, "Session 0x" UINT64_FORMAT " waiting for data\n", conn->session_id); 549 | all_idle = 0; 550 | break; 551 | case -1: 552 | /* error during select() */ 553 | rc = -1; 554 | default: 555 | /* read input from connection */ 556 | if (! PQconsumeInput(conn->db_conn)) { 557 | fprintf(stderr, "Error reading from database: %s\n", PQerrorMessage(conn->db_conn)); 558 | rc = -1; 559 | } else { 560 | /* check if we are done reading */ 561 | if (PQisBusy(conn->db_conn)) { 562 | /* more to read */ 563 | all_idle = 0; 564 | } else { 565 | /* read and discard all results */ 566 | while (NULL != (result = PQgetResult(conn->db_conn))) { 567 | /* count statements and errors for statistics */ 568 | ++stat_stmt; 569 | result_status = PQresultStatus(result); 570 | debug(2, "Session 0x" UINT64_FORMAT " got query response (%s)\n", 571 | conn->session_id, 572 | (PGRES_TUPLES_OK == result_status) ? "PGRES_TUPLES_OK" : 573 | ((PGRES_COMMAND_OK == result_status) ? "PGRES_COMMAND_OK" : 574 | ((PGRES_FATAL_ERROR == result_status) ? "PGRES_FATAL_ERROR" : 575 | ((PGRES_NONFATAL_ERROR == result_status) ? "PGRES_NONFATAL_ERROR" : 576 | ((PGRES_EMPTY_QUERY == result_status) ? "PGRES_EMPTY_QUERY" : "unexpected status"))))); 577 | 578 | if (PGRES_FATAL_ERROR == result_status) 579 | debug(1, "Error message session 0x" UINT64_FORMAT ": %s\n", 580 | conn->session_id, PQresultErrorMessage(result)); 581 | 582 | if ((PGRES_EMPTY_QUERY != result_status) 583 | && (PGRES_COMMAND_OK != result_status) 584 | && (PGRES_TUPLES_OK != result_status) 585 | && (PGRES_NONFATAL_ERROR != result_status)) 586 | { 587 | ++stat_errors; 588 | } 589 | 590 | PQclear(result); 591 | } 592 | 593 | /* one less concurrent statement */ 594 | --stat_statements; 595 | 596 | conn->status = idle; 597 | 598 | /* remember execution time for statistics */ 599 | if (-1 == gettimeofday(&delta, NULL)) { 600 | perror("Error calling gettimeofday"); 601 | rc = -1; 602 | } else { 603 | /* subtract statement start time */ 604 | timersub(&delta, &(conn->stmt_start), &delta); 605 | 606 | /* add to duration histogram */ 607 | if (0 == delta.tv_sec) { 608 | if (20000 >= delta.tv_usec) { 609 | ++stat_hist[0]; 610 | } else if (100000 >= delta.tv_usec) { 611 | ++stat_hist[1]; 612 | } else if (500000 >= delta.tv_usec) { 613 | ++stat_hist[2]; 614 | } else { 615 | ++stat_hist[3]; 616 | } 617 | } else if (2 > delta.tv_sec) { 618 | ++stat_hist[3]; 619 | } else { 620 | ++stat_hist[4]; 621 | } 622 | 623 | /* remember longest statement */ 624 | if ((delta.tv_sec > stat_longstmt.tv_sec) 625 | || ((delta.tv_sec == stat_longstmt.tv_sec) 626 | && (delta.tv_usec > stat_longstmt.tv_usec))) 627 | { 628 | stat_longstmt.tv_sec = delta.tv_sec; 629 | stat_longstmt.tv_usec = delta.tv_usec; 630 | } 631 | 632 | /* add to total */ 633 | timeradd(&stat_exec, &delta, &stat_exec); 634 | } 635 | } 636 | } 637 | break; 638 | } 639 | break; 640 | } 641 | 642 | if (! found_conn) { 643 | /* remember previous item in list, useful for removing an item */ 644 | prev_conn = conn; 645 | } 646 | 647 | conn = conn->next; 648 | } 649 | 650 | /* make sure we found a connection above (except for connect items) */ 651 | if (1 == rc) { 652 | if ((pg_connect == type) && (NULL != found_conn)) { 653 | fprintf(stderr, "Error: connection for session 0x" UINT64_FORMAT " already exists\n", replay_get_session_id(item)); 654 | rc = -1; 655 | } else if ((pg_connect != type) && (NULL == found_conn)) { 656 | fprintf(stderr, "Error: no connection found for session 0x" UINT64_FORMAT "\n", replay_get_session_id(item)); 657 | rc = -1; 658 | } 659 | } 660 | 661 | /* time when the statement originally ran */ 662 | stmt_time = replay_get_time(item); 663 | last_stmt_time.tv_sec = stmt_time->tv_sec; 664 | last_stmt_time.tv_usec = stmt_time->tv_usec; 665 | 666 | /* set first_stmt_time if it is not yet set */ 667 | if (! fstmtm_set) { 668 | first_stmt_time.tv_sec = stmt_time->tv_sec; 669 | first_stmt_time.tv_usec = stmt_time->tv_usec; 670 | 671 | fstmtm_set = 1; 672 | } 673 | 674 | /* get current time */ 675 | if (-1 != rc) { 676 | if (-1 == gettimeofday(&now, NULL)) { 677 | fprintf(stderr, "Error: gettimeofday failed\n"); 678 | rc = -1; 679 | } 680 | } 681 | 682 | /* determine if statement should already be consumed, sleep if necessary */ 683 | if (-1 != rc) { 684 | /* calculate "target time" when item should be replayed: 685 | statement time - first statement time 686 | program start time - skipped time + ------------------------------------- 687 | replay factor */ 688 | 689 | /* timestamp of the statement */ 690 | target_time.tv_sec = stmt_time->tv_sec; 691 | target_time.tv_usec = stmt_time->tv_usec; 692 | 693 | /* subtract time of first statement */ 694 | timersub(&target_time, &first_stmt_time, &target_time); 695 | 696 | /* subtract skipped time */ 697 | if (jump_enabled) { 698 | timersub(&target_time, &jump_total, &target_time); 699 | } 700 | 701 | /* divide by replay_factor */ 702 | if (replay_factor != 1.0) { 703 | /* - divide the seconds part by the factor 704 | - divide the microsecond part by the factor and add the 705 | fractional part (times 10^6) of the previous division 706 | - if the result exceeds 10^6, subtract the excess and 707 | add its 10^6th to the seconds part. */ 708 | d = target_time.tv_sec / replay_factor; 709 | target_time.tv_sec = d; 710 | target_time.tv_usec = target_time.tv_usec / replay_factor + 711 | (d - target_time.tv_sec) * 1000000.0; 712 | i = target_time.tv_usec / 1000000; 713 | target_time.tv_usec -= i * 1000000; 714 | target_time.tv_sec += i; 715 | } 716 | 717 | /* add program start time */ 718 | timeradd(&target_time, &start_time, &target_time); 719 | 720 | /* warn if we fall behind too much */ 721 | if (secs_behind < now.tv_sec - target_time.tv_sec) { 722 | secs_behind = now.tv_sec - target_time.tv_sec; 723 | for (j=0; j now.tv_sec) || 733 | ((target_time.tv_sec == now.tv_sec) && (target_time.tv_usec > now.tv_usec))) && 734 | all_idle) { 735 | /* sleep or jump if all is idle and the target time is in the future */ 736 | 737 | /* calculate time to sleep or jump (delta = target_time - now) */ 738 | timersub(&target_time, &now, &delta); 739 | 740 | if (jump_enabled) { 741 | /* add the sleep time to jump_total */ 742 | timeradd(&jump_total, &delta, &jump_total); 743 | debug(2, "Skipping %lu.%06lu seconds\n", (unsigned long)delta.tv_sec, (unsigned long)delta.tv_usec); 744 | /* then consume item */ 745 | rc = 1; 746 | } else { 747 | /* sleep */ 748 | if (-1 == do_sleep(&delta)) { 749 | rc = -1; 750 | } else { 751 | /* then consume item */ 752 | rc = 1; 753 | } 754 | } 755 | } else if (((target_time.tv_sec < now.tv_sec) || 756 | ((target_time.tv_sec == now.tv_sec) && (target_time.tv_usec <= now.tv_usec))) && 757 | ((pg_connect == type) || 758 | ((pg_disconnect == type) && (closed == found_conn->status)) || 759 | ((pg_cancel == type) && (wait_read == found_conn->status)) || 760 | (idle == found_conn->status))) { 761 | /* if the item is due and its connection is idle, consume it */ 762 | /* cancel items will also be consumed if the connection is waiting for a resonse */ 763 | rc = 1; 764 | } else if (found_conn && (closed == found_conn->status)) { 765 | fprintf(stderr, "Connection 0x" UINT64_FORMAT " failed with FATAL error: %s\n", 766 | found_conn->session_id, found_conn->errmsg); 767 | rc = -1; 768 | } 769 | } 770 | 771 | /* send statement */ 772 | if (1 == rc) { 773 | /* count for statistics */ 774 | ++stat_actions; 775 | 776 | switch (type) { 777 | case pg_connect: 778 | debug(2, "Starting database connection for session 0x" UINT64_FORMAT "\n", replay_get_session_id(item)); 779 | 780 | /* allocate a connect string */ 781 | user = replay_get_user(item); 782 | database = replay_get_database(item); 783 | if (NULL == (connstr = malloc(strlen(conn_string) + 2 * strlen(user) + 2 * strlen(database) + 18))) { 784 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)strlen(conn_string) + 2 * strlen(user) + 2 * strlen(database) + 18); 785 | rc = -1; 786 | } else { 787 | /* append user and password */ 788 | strcpy(connstr, conn_string); 789 | p1 = connstr + strlen(connstr); 790 | *(p1++) = ' '; 791 | *(p1++) = 'u'; 792 | *(p1++) = 's'; 793 | *(p1++) = 'e'; 794 | *(p1++) = 'r'; 795 | *(p1++) = '='; 796 | *(p1++) = '\''; 797 | for (p=user; '\0'!=*p; ++p) { 798 | if (('\'' == *p) || ('\\' == *p)) { 799 | *(p1++) = '\\'; 800 | } 801 | *(p1++) = *p; 802 | } 803 | *(p1++) = '\''; 804 | *(p1++) = ' '; 805 | *(p1++) = 'd'; 806 | *(p1++) = 'b'; 807 | *(p1++) = 'n'; 808 | *(p1++) = 'a'; 809 | *(p1++) = 'm'; 810 | *(p1++) = 'e'; 811 | *(p1++) = '='; 812 | *(p1++) = '\''; 813 | for (p=database; '\0'!=*p; ++p) { 814 | if (('\'' == *p) || ('\\' == *p)) { 815 | *(p1++) = '\\'; 816 | } 817 | *(p1++) = *p; 818 | } 819 | *(p1++) = '\''; 820 | *p1 = '\0'; 821 | 822 | /* allocate a struct dbconn */ 823 | if (NULL == (found_conn = malloc(sizeof(struct dbconn)))) { 824 | fprintf(stderr, "Cannot allocate %lu bytes of memory\n", (unsigned long)sizeof(struct dbconn)); 825 | rc = -1; 826 | } else { 827 | /* getaddrinfo() may set errno */ 828 | errno = 0; 829 | /* initialize a connection */ 830 | if (NULL == (found_conn->db_conn = PQconnectStart(connstr))) { 831 | fprintf(stderr, "Cannot allocate memory for database connection\n"); 832 | rc = -1; 833 | free(found_conn); 834 | } else { 835 | if (CONNECTION_BAD == PQstatus(found_conn->db_conn)) { 836 | fprintf(stderr, "Error: connection to database failed: %s\n", PQerrorMessage(found_conn->db_conn)); 837 | if (errno) 838 | fprintf(stderr, "Error detail: %s\n", strerror(errno)); 839 | rc = -1; 840 | PQfinish(found_conn->db_conn); 841 | free(found_conn); 842 | } else { 843 | if (-1 == (found_conn->socket = PQsocket(found_conn->db_conn))) { 844 | fprintf(stderr, "Error: cannot get socket for database connection\n"); 845 | rc = -1; 846 | PQfinish(found_conn->db_conn); 847 | free(found_conn); 848 | } else { 849 | /* set values in struct dbconn */ 850 | 851 | found_conn->session_id = replay_get_session_id(item); 852 | found_conn->status = conn_wait_write; 853 | found_conn->errmsg = NULL; 854 | found_conn->next = connections; 855 | 856 | connections = found_conn; 857 | 858 | /* do not display notices */ 859 | PQsetNoticeReceiver(found_conn->db_conn, ignore_notices, NULL); 860 | } 861 | } 862 | } 863 | } 864 | 865 | /* free connection sting */ 866 | free(connstr); 867 | } 868 | break; 869 | case pg_disconnect: 870 | /* dead connections need not be closed */ 871 | if (closed == found_conn->status) { 872 | debug(2, "Removing closed session 0x" UINT64_FORMAT "\n", replay_get_session_id(item)); 873 | } else { 874 | debug(2, "Disconnecting database connection for session 0x" UINT64_FORMAT "\n", replay_get_session_id(item)); 875 | 876 | PQfinish(found_conn->db_conn); 877 | 878 | /* remember session duration for statistics */ 879 | if (-1 == gettimeofday(&delta, NULL)) { 880 | perror("Error calling gettimeofday"); 881 | rc = -1; 882 | } else { 883 | /* subtract session start time */ 884 | timersub(&delta, &(found_conn->session_start), &delta); 885 | 886 | /* add to total */ 887 | timeradd(&stat_session, &delta, &stat_session); 888 | } 889 | 890 | /* one less concurrent session */ 891 | --stat_sessions; 892 | } 893 | 894 | /* remove struct dbconn from linked list */ 895 | if (prev_conn) { 896 | prev_conn->next = found_conn->next; 897 | } else { 898 | connections = found_conn->next; 899 | } 900 | if (found_conn->errmsg) { 901 | free(found_conn->errmsg); 902 | } 903 | free(found_conn); 904 | 905 | break; 906 | case pg_execute: 907 | debug(2, "Sending simple statement on session 0x" UINT64_FORMAT "\n", replay_get_session_id(item)); 908 | 909 | if (! PQsendQuery(found_conn->db_conn, replay_get_statement(item))) { 910 | fprintf(stderr, "Error sending simple statement: %s\n", PQerrorMessage(found_conn->db_conn)); 911 | rc = -1; 912 | } else { 913 | found_conn->status = wait_write; 914 | } 915 | break; 916 | case pg_prepare: 917 | debug(2, "Sending prepare request on session 0x" UINT64_FORMAT "\n", replay_get_session_id(item)); 918 | 919 | /* count preparations for statistics */ 920 | ++stat_prep; 921 | 922 | if (! PQsendPrepare( 923 | found_conn->db_conn, 924 | replay_get_name(item), 925 | replay_get_statement(item), 926 | 0, 927 | NULL)) { 928 | fprintf(stderr, "Error sending prepare request: %s\n", PQerrorMessage(found_conn->db_conn)); 929 | rc = -1; 930 | } else { 931 | found_conn->status = wait_write; 932 | } 933 | break; 934 | case pg_exec_prepared: 935 | debug(2, "Sending prepared statement execution on session 0x" UINT64_FORMAT "\n", replay_get_session_id(item)); 936 | 937 | if (! PQsendQueryPrepared( 938 | found_conn->db_conn, 939 | replay_get_name(item), 940 | replay_get_valuecount(item), 941 | replay_get_values(item), 942 | NULL, 943 | NULL, 944 | 0)) { 945 | fprintf(stderr, "Error sending prepared statement execution: %s\n", PQerrorMessage(found_conn->db_conn)); 946 | rc = -1; 947 | } else { 948 | found_conn->status = wait_write; 949 | } 950 | break; 951 | case pg_cancel: 952 | debug(2, "Sending cancel request on session 0x" UINT64_FORMAT "\n", replay_get_session_id(item)); 953 | 954 | if (NULL == (cancel_request = PQgetCancel(found_conn->db_conn))) { 955 | fprintf(stderr, "Error creating cancel request\n"); 956 | rc = -1; 957 | } else { 958 | if (! PQcancel(cancel_request, errbuf, 256)) { 959 | fprintf(stderr, "Error sending cancel request: %s\n", errbuf); 960 | rc = -1; 961 | } 962 | /* free cancel request */ 963 | PQfreeCancel(cancel_request); 964 | } 965 | /* status remains unchanged */ 966 | break; 967 | } 968 | 969 | replay_free(item); 970 | } 971 | 972 | /* try to flush the statement if necessary */ 973 | if ((1 == rc) && (pg_disconnect != type) && (wait_write == found_conn->status)) { 974 | switch (PQflush(found_conn->db_conn)) { 975 | case 0: 976 | /* complete request sent */ 977 | found_conn->status = wait_read; 978 | break; 979 | case 1: 980 | debug(2, "Session 0x" UINT64_FORMAT " needs to flush again\n", found_conn->session_id); 981 | break; 982 | default: 983 | fprintf(stderr, "Error flushing to database: %s\n", PQerrorMessage(found_conn->db_conn)); 984 | rc = -1; 985 | } 986 | 987 | /* get statement start time */ 988 | if (-1 == gettimeofday(&(found_conn->stmt_start), NULL)) { 989 | perror("Error calling gettimeofday"); 990 | rc = -1; 991 | } 992 | 993 | /* count concurrent statements */ 994 | if (++stat_statements > stat_stmtmax) { 995 | stat_stmtmax = stat_statements; 996 | } 997 | } 998 | 999 | debug(3, "Leaving database_consumer%s\n", ""); 1000 | return rc; 1001 | } 1002 | 1003 | int database_consumer_dry_run(replay_item *item) { 1004 | const replay_type type = replay_get_type(item); 1005 | debug(3, "Entering database_consumer_dry_run%s\n", ""); 1006 | const struct timeval *stmt_time; 1007 | static int fstmt_set_dr = 0; 1008 | 1009 | /* time when the statement originally ran */ 1010 | stmt_time = replay_get_time(item); 1011 | last_stmt_time.tv_sec = stmt_time->tv_sec; 1012 | last_stmt_time.tv_usec = stmt_time->tv_usec; 1013 | 1014 | /* set first_stmt_time if it is not yet set */ 1015 | if (! fstmt_set_dr) { 1016 | first_stmt_time.tv_sec = stmt_time->tv_sec; 1017 | first_stmt_time.tv_usec = stmt_time->tv_usec; 1018 | 1019 | fstmt_set_dr = 1; 1020 | } 1021 | 1022 | /* gather statistics */ 1023 | ++stat_actions; 1024 | 1025 | switch (type) { 1026 | case pg_connect: 1027 | ++stat_sesscnt; 1028 | if (++stat_sessions > stat_sessmax) { 1029 | stat_sessmax = stat_sessions; 1030 | } 1031 | break; 1032 | case pg_disconnect: 1033 | --stat_sessions; 1034 | break; 1035 | case pg_execute: 1036 | case pg_exec_prepared: 1037 | ++stat_stmt; 1038 | break; 1039 | case pg_prepare: 1040 | ++stat_prep; 1041 | break; 1042 | case pg_cancel: 1043 | break; 1044 | } 1045 | 1046 | replay_free(item); 1047 | debug(3, "Leaving database_consumer_dry_run%s\n", ""); 1048 | 1049 | return 1; 1050 | } 1051 | -------------------------------------------------------------------------------- /config.sub: -------------------------------------------------------------------------------- 1 | #! /bin/sh 2 | # Configuration validation subroutine script. 3 | # Copyright 1992-2017 Free Software Foundation, Inc. 4 | 5 | timestamp='2017-11-23' 6 | 7 | # This file is free software; you can redistribute it and/or modify it 8 | # under the terms of the GNU General Public License as published by 9 | # the Free Software Foundation; either version 3 of the License, or 10 | # (at your option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, but 13 | # WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 15 | # General Public License for more details. 16 | # 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program; if not, see . 19 | # 20 | # As a special exception to the GNU General Public License, if you 21 | # distribute this file as part of a program that contains a 22 | # configuration script generated by Autoconf, you may include it under 23 | # the same distribution terms that you use for the rest of that 24 | # program. This Exception is an additional permission under section 7 25 | # of the GNU General Public License, version 3 ("GPLv3"). 26 | 27 | 28 | # Please send patches to . 29 | # 30 | # Configuration subroutine to validate and canonicalize a configuration type. 31 | # Supply the specified configuration type as an argument. 32 | # If it is invalid, we print an error message on stderr and exit with code 1. 33 | # Otherwise, we print the canonical config type on stdout and succeed. 34 | 35 | # You can get the latest version of this script from: 36 | # https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub 37 | 38 | # This file is supposed to be the same for all GNU packages 39 | # and recognize all the CPU types, system types and aliases 40 | # that are meaningful with *any* GNU software. 41 | # Each package is responsible for reporting which valid configurations 42 | # it does not support. The user should be able to distinguish 43 | # a failure to support a valid configuration from a meaningless 44 | # configuration. 45 | 46 | # The goal of this file is to map all the various variations of a given 47 | # machine specification into a single specification in the form: 48 | # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM 49 | # or in some cases, the newer four-part form: 50 | # CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM 51 | # It is wrong to echo any other type of specification. 52 | 53 | me=`echo "$0" | sed -e 's,.*/,,'` 54 | 55 | usage="\ 56 | Usage: $0 [OPTION] CPU-MFR-OPSYS or ALIAS 57 | 58 | Canonicalize a configuration name. 59 | 60 | Options: 61 | -h, --help print this help, then exit 62 | -t, --time-stamp print date of last modification, then exit 63 | -v, --version print version number, then exit 64 | 65 | Report bugs and patches to ." 66 | 67 | version="\ 68 | GNU config.sub ($timestamp) 69 | 70 | Copyright 1992-2017 Free Software Foundation, Inc. 71 | 72 | This is free software; see the source for copying conditions. There is NO 73 | warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." 74 | 75 | help=" 76 | Try \`$me --help' for more information." 77 | 78 | # Parse command line 79 | while test $# -gt 0 ; do 80 | case $1 in 81 | --time-stamp | --time* | -t ) 82 | echo "$timestamp" ; exit ;; 83 | --version | -v ) 84 | echo "$version" ; exit ;; 85 | --help | --h* | -h ) 86 | echo "$usage"; exit ;; 87 | -- ) # Stop option processing 88 | shift; break ;; 89 | - ) # Use stdin as input. 90 | break ;; 91 | -* ) 92 | echo "$me: invalid option $1$help" 93 | exit 1 ;; 94 | 95 | *local*) 96 | # First pass through any local machine types. 97 | echo $1 98 | exit ;; 99 | 100 | * ) 101 | break ;; 102 | esac 103 | done 104 | 105 | case $# in 106 | 0) echo "$me: missing argument$help" >&2 107 | exit 1;; 108 | 1) ;; 109 | *) echo "$me: too many arguments$help" >&2 110 | exit 1;; 111 | esac 112 | 113 | # Separate what the user gave into CPU-COMPANY and OS or KERNEL-OS (if any). 114 | # Here we must recognize all the valid KERNEL-OS combinations. 115 | maybe_os=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\2/'` 116 | case $maybe_os in 117 | nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc | linux-newlib* | \ 118 | linux-musl* | linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | \ 119 | knetbsd*-gnu* | netbsd*-gnu* | netbsd*-eabi* | \ 120 | kopensolaris*-gnu* | cloudabi*-eabi* | \ 121 | storm-chaos* | os2-emx* | rtmk-nova*) 122 | os=-$maybe_os 123 | basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'` 124 | ;; 125 | android-linux) 126 | os=-linux-android 127 | basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`-unknown 128 | ;; 129 | *) 130 | basic_machine=`echo $1 | sed 's/-[^-]*$//'` 131 | if [ $basic_machine != $1 ] 132 | then os=`echo $1 | sed 's/.*-/-/'` 133 | else os=; fi 134 | ;; 135 | esac 136 | 137 | ### Let's recognize common machines as not being operating systems so 138 | ### that things like config.sub decstation-3100 work. We also 139 | ### recognize some manufacturers as not being operating systems, so we 140 | ### can provide default operating systems below. 141 | case $os in 142 | -sun*os*) 143 | # Prevent following clause from handling this invalid input. 144 | ;; 145 | -dec* | -mips* | -sequent* | -encore* | -pc532* | -sgi* | -sony* | \ 146 | -att* | -7300* | -3300* | -delta* | -motorola* | -sun[234]* | \ 147 | -unicom* | -ibm* | -next | -hp | -isi* | -apollo | -altos* | \ 148 | -convergent* | -ncr* | -news | -32* | -3600* | -3100* | -hitachi* |\ 149 | -c[123]* | -convex* | -sun | -crds | -omron* | -dg | -ultra | -tti* | \ 150 | -harris | -dolphin | -highlevel | -gould | -cbm | -ns | -masscomp | \ 151 | -apple | -axis | -knuth | -cray | -microblaze*) 152 | os= 153 | basic_machine=$1 154 | ;; 155 | -bluegene*) 156 | os=-cnk 157 | ;; 158 | -sim | -cisco | -oki | -wec | -winbond) 159 | os= 160 | basic_machine=$1 161 | ;; 162 | -scout) 163 | ;; 164 | -wrs) 165 | os=-vxworks 166 | basic_machine=$1 167 | ;; 168 | -chorusos*) 169 | os=-chorusos 170 | basic_machine=$1 171 | ;; 172 | -chorusrdb) 173 | os=-chorusrdb 174 | basic_machine=$1 175 | ;; 176 | -hiux*) 177 | os=-hiuxwe2 178 | ;; 179 | -sco6) 180 | os=-sco5v6 181 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 182 | ;; 183 | -sco5) 184 | os=-sco3.2v5 185 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 186 | ;; 187 | -sco4) 188 | os=-sco3.2v4 189 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 190 | ;; 191 | -sco3.2.[4-9]*) 192 | os=`echo $os | sed -e 's/sco3.2./sco3.2v/'` 193 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 194 | ;; 195 | -sco3.2v[4-9]*) 196 | # Don't forget version if it is 3.2v4 or newer. 197 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 198 | ;; 199 | -sco5v6*) 200 | # Don't forget version if it is 3.2v4 or newer. 201 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 202 | ;; 203 | -sco*) 204 | os=-sco3.2v2 205 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 206 | ;; 207 | -udk*) 208 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 209 | ;; 210 | -isc) 211 | os=-isc2.2 212 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 213 | ;; 214 | -clix*) 215 | basic_machine=clipper-intergraph 216 | ;; 217 | -isc*) 218 | basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` 219 | ;; 220 | -lynx*178) 221 | os=-lynxos178 222 | ;; 223 | -lynx*5) 224 | os=-lynxos5 225 | ;; 226 | -lynx*) 227 | os=-lynxos 228 | ;; 229 | -ptx*) 230 | basic_machine=`echo $1 | sed -e 's/86-.*/86-sequent/'` 231 | ;; 232 | -psos*) 233 | os=-psos 234 | ;; 235 | -mint | -mint[0-9]*) 236 | basic_machine=m68k-atari 237 | os=-mint 238 | ;; 239 | esac 240 | 241 | # Decode aliases for certain CPU-COMPANY combinations. 242 | case $basic_machine in 243 | # Recognize the basic CPU types without company name. 244 | # Some are omitted here because they have special meanings below. 245 | 1750a | 580 \ 246 | | a29k \ 247 | | aarch64 | aarch64_be \ 248 | | alpha | alphaev[4-8] | alphaev56 | alphaev6[78] | alphapca5[67] \ 249 | | alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] | alpha64pca5[67] \ 250 | | am33_2.0 \ 251 | | arc | arceb \ 252 | | arm | arm[bl]e | arme[lb] | armv[2-8] | armv[3-8][lb] | armv7[arm] \ 253 | | avr | avr32 \ 254 | | ba \ 255 | | be32 | be64 \ 256 | | bfin \ 257 | | c4x | c8051 | clipper \ 258 | | d10v | d30v | dlx | dsp16xx \ 259 | | e2k | epiphany \ 260 | | fido | fr30 | frv | ft32 \ 261 | | h8300 | h8500 | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \ 262 | | hexagon \ 263 | | i370 | i860 | i960 | ia16 | ia64 \ 264 | | ip2k | iq2000 \ 265 | | k1om \ 266 | | le32 | le64 \ 267 | | lm32 \ 268 | | m32c | m32r | m32rle | m68000 | m68k | m88k \ 269 | | maxq | mb | microblaze | microblazeel | mcore | mep | metag \ 270 | | mips | mipsbe | mipseb | mipsel | mipsle \ 271 | | mips16 \ 272 | | mips64 | mips64el \ 273 | | mips64octeon | mips64octeonel \ 274 | | mips64orion | mips64orionel \ 275 | | mips64r5900 | mips64r5900el \ 276 | | mips64vr | mips64vrel \ 277 | | mips64vr4100 | mips64vr4100el \ 278 | | mips64vr4300 | mips64vr4300el \ 279 | | mips64vr5000 | mips64vr5000el \ 280 | | mips64vr5900 | mips64vr5900el \ 281 | | mipsisa32 | mipsisa32el \ 282 | | mipsisa32r2 | mipsisa32r2el \ 283 | | mipsisa32r6 | mipsisa32r6el \ 284 | | mipsisa64 | mipsisa64el \ 285 | | mipsisa64r2 | mipsisa64r2el \ 286 | | mipsisa64r6 | mipsisa64r6el \ 287 | | mipsisa64sb1 | mipsisa64sb1el \ 288 | | mipsisa64sr71k | mipsisa64sr71kel \ 289 | | mipsr5900 | mipsr5900el \ 290 | | mipstx39 | mipstx39el \ 291 | | mn10200 | mn10300 \ 292 | | moxie \ 293 | | mt \ 294 | | msp430 \ 295 | | nds32 | nds32le | nds32be \ 296 | | nios | nios2 | nios2eb | nios2el \ 297 | | ns16k | ns32k \ 298 | | open8 | or1k | or1knd | or32 \ 299 | | pdp10 | pdp11 | pj | pjl \ 300 | | powerpc | powerpc64 | powerpc64le | powerpcle \ 301 | | pru \ 302 | | pyramid \ 303 | | riscv32 | riscv64 \ 304 | | rl78 | rx \ 305 | | score \ 306 | | sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[234]eb | sheb | shbe | shle | sh[1234]le | sh3ele \ 307 | | sh64 | sh64le \ 308 | | sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \ 309 | | sparcv8 | sparcv9 | sparcv9b | sparcv9v \ 310 | | spu \ 311 | | tahoe | tic4x | tic54x | tic55x | tic6x | tic80 | tron \ 312 | | ubicom32 \ 313 | | v850 | v850e | v850e1 | v850e2 | v850es | v850e2v3 \ 314 | | visium \ 315 | | wasm32 \ 316 | | x86 | xc16x | xstormy16 | xtensa \ 317 | | z8k | z80) 318 | basic_machine=$basic_machine-unknown 319 | ;; 320 | c54x) 321 | basic_machine=tic54x-unknown 322 | ;; 323 | c55x) 324 | basic_machine=tic55x-unknown 325 | ;; 326 | c6x) 327 | basic_machine=tic6x-unknown 328 | ;; 329 | leon|leon[3-9]) 330 | basic_machine=sparc-$basic_machine 331 | ;; 332 | m6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x | nvptx | picochip) 333 | basic_machine=$basic_machine-unknown 334 | os=-none 335 | ;; 336 | m88110 | m680[12346]0 | m683?2 | m68360 | m5200 | v70 | w65 | z8k) 337 | ;; 338 | ms1) 339 | basic_machine=mt-unknown 340 | ;; 341 | 342 | strongarm | thumb | xscale) 343 | basic_machine=arm-unknown 344 | ;; 345 | xgate) 346 | basic_machine=$basic_machine-unknown 347 | os=-none 348 | ;; 349 | xscaleeb) 350 | basic_machine=armeb-unknown 351 | ;; 352 | 353 | xscaleel) 354 | basic_machine=armel-unknown 355 | ;; 356 | 357 | # We use `pc' rather than `unknown' 358 | # because (1) that's what they normally are, and 359 | # (2) the word "unknown" tends to confuse beginning users. 360 | i*86 | x86_64) 361 | basic_machine=$basic_machine-pc 362 | ;; 363 | # Object if more than one company name word. 364 | *-*-*) 365 | echo Invalid configuration \`$1\': machine \`$basic_machine\' not recognized 1>&2 366 | exit 1 367 | ;; 368 | # Recognize the basic CPU types with company name. 369 | 580-* \ 370 | | a29k-* \ 371 | | aarch64-* | aarch64_be-* \ 372 | | alpha-* | alphaev[4-8]-* | alphaev56-* | alphaev6[78]-* \ 373 | | alpha64-* | alpha64ev[4-8]-* | alpha64ev56-* | alpha64ev6[78]-* \ 374 | | alphapca5[67]-* | alpha64pca5[67]-* | arc-* | arceb-* \ 375 | | arm-* | armbe-* | armle-* | armeb-* | armv*-* \ 376 | | avr-* | avr32-* \ 377 | | ba-* \ 378 | | be32-* | be64-* \ 379 | | bfin-* | bs2000-* \ 380 | | c[123]* | c30-* | [cjt]90-* | c4x-* \ 381 | | c8051-* | clipper-* | craynv-* | cydra-* \ 382 | | d10v-* | d30v-* | dlx-* \ 383 | | e2k-* | elxsi-* \ 384 | | f30[01]-* | f700-* | fido-* | fr30-* | frv-* | fx80-* \ 385 | | h8300-* | h8500-* \ 386 | | hppa-* | hppa1.[01]-* | hppa2.0-* | hppa2.0[nw]-* | hppa64-* \ 387 | | hexagon-* \ 388 | | i*86-* | i860-* | i960-* | ia16-* | ia64-* \ 389 | | ip2k-* | iq2000-* \ 390 | | k1om-* \ 391 | | le32-* | le64-* \ 392 | | lm32-* \ 393 | | m32c-* | m32r-* | m32rle-* \ 394 | | m68000-* | m680[012346]0-* | m68360-* | m683?2-* | m68k-* \ 395 | | m88110-* | m88k-* | maxq-* | mcore-* | metag-* \ 396 | | microblaze-* | microblazeel-* \ 397 | | mips-* | mipsbe-* | mipseb-* | mipsel-* | mipsle-* \ 398 | | mips16-* \ 399 | | mips64-* | mips64el-* \ 400 | | mips64octeon-* | mips64octeonel-* \ 401 | | mips64orion-* | mips64orionel-* \ 402 | | mips64r5900-* | mips64r5900el-* \ 403 | | mips64vr-* | mips64vrel-* \ 404 | | mips64vr4100-* | mips64vr4100el-* \ 405 | | mips64vr4300-* | mips64vr4300el-* \ 406 | | mips64vr5000-* | mips64vr5000el-* \ 407 | | mips64vr5900-* | mips64vr5900el-* \ 408 | | mipsisa32-* | mipsisa32el-* \ 409 | | mipsisa32r2-* | mipsisa32r2el-* \ 410 | | mipsisa32r6-* | mipsisa32r6el-* \ 411 | | mipsisa64-* | mipsisa64el-* \ 412 | | mipsisa64r2-* | mipsisa64r2el-* \ 413 | | mipsisa64r6-* | mipsisa64r6el-* \ 414 | | mipsisa64sb1-* | mipsisa64sb1el-* \ 415 | | mipsisa64sr71k-* | mipsisa64sr71kel-* \ 416 | | mipsr5900-* | mipsr5900el-* \ 417 | | mipstx39-* | mipstx39el-* \ 418 | | mmix-* \ 419 | | mt-* \ 420 | | msp430-* \ 421 | | nds32-* | nds32le-* | nds32be-* \ 422 | | nios-* | nios2-* | nios2eb-* | nios2el-* \ 423 | | none-* | np1-* | ns16k-* | ns32k-* \ 424 | | open8-* \ 425 | | or1k*-* \ 426 | | orion-* \ 427 | | pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \ 428 | | powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* \ 429 | | pru-* \ 430 | | pyramid-* \ 431 | | riscv32-* | riscv64-* \ 432 | | rl78-* | romp-* | rs6000-* | rx-* \ 433 | | sh-* | sh[1234]-* | sh[24]a-* | sh[24]aeb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \ 434 | | shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \ 435 | | sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \ 436 | | sparclite-* \ 437 | | sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx*-* \ 438 | | tahoe-* \ 439 | | tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \ 440 | | tile*-* \ 441 | | tron-* \ 442 | | ubicom32-* \ 443 | | v850-* | v850e-* | v850e1-* | v850es-* | v850e2-* | v850e2v3-* \ 444 | | vax-* \ 445 | | visium-* \ 446 | | wasm32-* \ 447 | | we32k-* \ 448 | | x86-* | x86_64-* | xc16x-* | xps100-* \ 449 | | xstormy16-* | xtensa*-* \ 450 | | ymp-* \ 451 | | z8k-* | z80-*) 452 | ;; 453 | # Recognize the basic CPU types without company name, with glob match. 454 | xtensa*) 455 | basic_machine=$basic_machine-unknown 456 | ;; 457 | # Recognize the various machine names and aliases which stand 458 | # for a CPU type and a company and sometimes even an OS. 459 | 386bsd) 460 | basic_machine=i386-unknown 461 | os=-bsd 462 | ;; 463 | 3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc) 464 | basic_machine=m68000-att 465 | ;; 466 | 3b*) 467 | basic_machine=we32k-att 468 | ;; 469 | a29khif) 470 | basic_machine=a29k-amd 471 | os=-udi 472 | ;; 473 | abacus) 474 | basic_machine=abacus-unknown 475 | ;; 476 | adobe68k) 477 | basic_machine=m68010-adobe 478 | os=-scout 479 | ;; 480 | alliant | fx80) 481 | basic_machine=fx80-alliant 482 | ;; 483 | altos | altos3068) 484 | basic_machine=m68k-altos 485 | ;; 486 | am29k) 487 | basic_machine=a29k-none 488 | os=-bsd 489 | ;; 490 | amd64) 491 | basic_machine=x86_64-pc 492 | ;; 493 | amd64-*) 494 | basic_machine=x86_64-`echo $basic_machine | sed 's/^[^-]*-//'` 495 | ;; 496 | amdahl) 497 | basic_machine=580-amdahl 498 | os=-sysv 499 | ;; 500 | amiga | amiga-*) 501 | basic_machine=m68k-unknown 502 | ;; 503 | amigaos | amigados) 504 | basic_machine=m68k-unknown 505 | os=-amigaos 506 | ;; 507 | amigaunix | amix) 508 | basic_machine=m68k-unknown 509 | os=-sysv4 510 | ;; 511 | apollo68) 512 | basic_machine=m68k-apollo 513 | os=-sysv 514 | ;; 515 | apollo68bsd) 516 | basic_machine=m68k-apollo 517 | os=-bsd 518 | ;; 519 | aros) 520 | basic_machine=i386-pc 521 | os=-aros 522 | ;; 523 | asmjs) 524 | basic_machine=asmjs-unknown 525 | ;; 526 | aux) 527 | basic_machine=m68k-apple 528 | os=-aux 529 | ;; 530 | balance) 531 | basic_machine=ns32k-sequent 532 | os=-dynix 533 | ;; 534 | blackfin) 535 | basic_machine=bfin-unknown 536 | os=-linux 537 | ;; 538 | blackfin-*) 539 | basic_machine=bfin-`echo $basic_machine | sed 's/^[^-]*-//'` 540 | os=-linux 541 | ;; 542 | bluegene*) 543 | basic_machine=powerpc-ibm 544 | os=-cnk 545 | ;; 546 | c54x-*) 547 | basic_machine=tic54x-`echo $basic_machine | sed 's/^[^-]*-//'` 548 | ;; 549 | c55x-*) 550 | basic_machine=tic55x-`echo $basic_machine | sed 's/^[^-]*-//'` 551 | ;; 552 | c6x-*) 553 | basic_machine=tic6x-`echo $basic_machine | sed 's/^[^-]*-//'` 554 | ;; 555 | c90) 556 | basic_machine=c90-cray 557 | os=-unicos 558 | ;; 559 | cegcc) 560 | basic_machine=arm-unknown 561 | os=-cegcc 562 | ;; 563 | convex-c1) 564 | basic_machine=c1-convex 565 | os=-bsd 566 | ;; 567 | convex-c2) 568 | basic_machine=c2-convex 569 | os=-bsd 570 | ;; 571 | convex-c32) 572 | basic_machine=c32-convex 573 | os=-bsd 574 | ;; 575 | convex-c34) 576 | basic_machine=c34-convex 577 | os=-bsd 578 | ;; 579 | convex-c38) 580 | basic_machine=c38-convex 581 | os=-bsd 582 | ;; 583 | cray | j90) 584 | basic_machine=j90-cray 585 | os=-unicos 586 | ;; 587 | craynv) 588 | basic_machine=craynv-cray 589 | os=-unicosmp 590 | ;; 591 | cr16 | cr16-*) 592 | basic_machine=cr16-unknown 593 | os=-elf 594 | ;; 595 | crds | unos) 596 | basic_machine=m68k-crds 597 | ;; 598 | crisv32 | crisv32-* | etraxfs*) 599 | basic_machine=crisv32-axis 600 | ;; 601 | cris | cris-* | etrax*) 602 | basic_machine=cris-axis 603 | ;; 604 | crx) 605 | basic_machine=crx-unknown 606 | os=-elf 607 | ;; 608 | da30 | da30-*) 609 | basic_machine=m68k-da30 610 | ;; 611 | decstation | decstation-3100 | pmax | pmax-* | pmin | dec3100 | decstatn) 612 | basic_machine=mips-dec 613 | ;; 614 | decsystem10* | dec10*) 615 | basic_machine=pdp10-dec 616 | os=-tops10 617 | ;; 618 | decsystem20* | dec20*) 619 | basic_machine=pdp10-dec 620 | os=-tops20 621 | ;; 622 | delta | 3300 | motorola-3300 | motorola-delta \ 623 | | 3300-motorola | delta-motorola) 624 | basic_machine=m68k-motorola 625 | ;; 626 | delta88) 627 | basic_machine=m88k-motorola 628 | os=-sysv3 629 | ;; 630 | dicos) 631 | basic_machine=i686-pc 632 | os=-dicos 633 | ;; 634 | djgpp) 635 | basic_machine=i586-pc 636 | os=-msdosdjgpp 637 | ;; 638 | dpx20 | dpx20-*) 639 | basic_machine=rs6000-bull 640 | os=-bosx 641 | ;; 642 | dpx2*) 643 | basic_machine=m68k-bull 644 | os=-sysv3 645 | ;; 646 | e500v[12]) 647 | basic_machine=powerpc-unknown 648 | os=$os"spe" 649 | ;; 650 | e500v[12]-*) 651 | basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'` 652 | os=$os"spe" 653 | ;; 654 | ebmon29k) 655 | basic_machine=a29k-amd 656 | os=-ebmon 657 | ;; 658 | elxsi) 659 | basic_machine=elxsi-elxsi 660 | os=-bsd 661 | ;; 662 | encore | umax | mmax) 663 | basic_machine=ns32k-encore 664 | ;; 665 | es1800 | OSE68k | ose68k | ose | OSE) 666 | basic_machine=m68k-ericsson 667 | os=-ose 668 | ;; 669 | fx2800) 670 | basic_machine=i860-alliant 671 | ;; 672 | genix) 673 | basic_machine=ns32k-ns 674 | ;; 675 | gmicro) 676 | basic_machine=tron-gmicro 677 | os=-sysv 678 | ;; 679 | go32) 680 | basic_machine=i386-pc 681 | os=-go32 682 | ;; 683 | h3050r* | hiux*) 684 | basic_machine=hppa1.1-hitachi 685 | os=-hiuxwe2 686 | ;; 687 | h8300hms) 688 | basic_machine=h8300-hitachi 689 | os=-hms 690 | ;; 691 | h8300xray) 692 | basic_machine=h8300-hitachi 693 | os=-xray 694 | ;; 695 | h8500hms) 696 | basic_machine=h8500-hitachi 697 | os=-hms 698 | ;; 699 | harris) 700 | basic_machine=m88k-harris 701 | os=-sysv3 702 | ;; 703 | hp300-*) 704 | basic_machine=m68k-hp 705 | ;; 706 | hp300bsd) 707 | basic_machine=m68k-hp 708 | os=-bsd 709 | ;; 710 | hp300hpux) 711 | basic_machine=m68k-hp 712 | os=-hpux 713 | ;; 714 | hp3k9[0-9][0-9] | hp9[0-9][0-9]) 715 | basic_machine=hppa1.0-hp 716 | ;; 717 | hp9k2[0-9][0-9] | hp9k31[0-9]) 718 | basic_machine=m68000-hp 719 | ;; 720 | hp9k3[2-9][0-9]) 721 | basic_machine=m68k-hp 722 | ;; 723 | hp9k6[0-9][0-9] | hp6[0-9][0-9]) 724 | basic_machine=hppa1.0-hp 725 | ;; 726 | hp9k7[0-79][0-9] | hp7[0-79][0-9]) 727 | basic_machine=hppa1.1-hp 728 | ;; 729 | hp9k78[0-9] | hp78[0-9]) 730 | # FIXME: really hppa2.0-hp 731 | basic_machine=hppa1.1-hp 732 | ;; 733 | hp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893) 734 | # FIXME: really hppa2.0-hp 735 | basic_machine=hppa1.1-hp 736 | ;; 737 | hp9k8[0-9][13679] | hp8[0-9][13679]) 738 | basic_machine=hppa1.1-hp 739 | ;; 740 | hp9k8[0-9][0-9] | hp8[0-9][0-9]) 741 | basic_machine=hppa1.0-hp 742 | ;; 743 | hppa-next) 744 | os=-nextstep3 745 | ;; 746 | hppaosf) 747 | basic_machine=hppa1.1-hp 748 | os=-osf 749 | ;; 750 | hppro) 751 | basic_machine=hppa1.1-hp 752 | os=-proelf 753 | ;; 754 | i370-ibm* | ibm*) 755 | basic_machine=i370-ibm 756 | ;; 757 | i*86v32) 758 | basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` 759 | os=-sysv32 760 | ;; 761 | i*86v4*) 762 | basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` 763 | os=-sysv4 764 | ;; 765 | i*86v) 766 | basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` 767 | os=-sysv 768 | ;; 769 | i*86sol2) 770 | basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` 771 | os=-solaris2 772 | ;; 773 | i386mach) 774 | basic_machine=i386-mach 775 | os=-mach 776 | ;; 777 | i386-vsta | vsta) 778 | basic_machine=i386-unknown 779 | os=-vsta 780 | ;; 781 | iris | iris4d) 782 | basic_machine=mips-sgi 783 | case $os in 784 | -irix*) 785 | ;; 786 | *) 787 | os=-irix4 788 | ;; 789 | esac 790 | ;; 791 | isi68 | isi) 792 | basic_machine=m68k-isi 793 | os=-sysv 794 | ;; 795 | leon-*|leon[3-9]-*) 796 | basic_machine=sparc-`echo $basic_machine | sed 's/-.*//'` 797 | ;; 798 | m68knommu) 799 | basic_machine=m68k-unknown 800 | os=-linux 801 | ;; 802 | m68knommu-*) 803 | basic_machine=m68k-`echo $basic_machine | sed 's/^[^-]*-//'` 804 | os=-linux 805 | ;; 806 | m88k-omron*) 807 | basic_machine=m88k-omron 808 | ;; 809 | magnum | m3230) 810 | basic_machine=mips-mips 811 | os=-sysv 812 | ;; 813 | merlin) 814 | basic_machine=ns32k-utek 815 | os=-sysv 816 | ;; 817 | microblaze*) 818 | basic_machine=microblaze-xilinx 819 | ;; 820 | mingw64) 821 | basic_machine=x86_64-pc 822 | os=-mingw64 823 | ;; 824 | mingw32) 825 | basic_machine=i686-pc 826 | os=-mingw32 827 | ;; 828 | mingw32ce) 829 | basic_machine=arm-unknown 830 | os=-mingw32ce 831 | ;; 832 | miniframe) 833 | basic_machine=m68000-convergent 834 | ;; 835 | *mint | -mint[0-9]* | *MiNT | *MiNT[0-9]*) 836 | basic_machine=m68k-atari 837 | os=-mint 838 | ;; 839 | mips3*-*) 840 | basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'` 841 | ;; 842 | mips3*) 843 | basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'`-unknown 844 | ;; 845 | monitor) 846 | basic_machine=m68k-rom68k 847 | os=-coff 848 | ;; 849 | morphos) 850 | basic_machine=powerpc-unknown 851 | os=-morphos 852 | ;; 853 | moxiebox) 854 | basic_machine=moxie-unknown 855 | os=-moxiebox 856 | ;; 857 | msdos) 858 | basic_machine=i386-pc 859 | os=-msdos 860 | ;; 861 | ms1-*) 862 | basic_machine=`echo $basic_machine | sed -e 's/ms1-/mt-/'` 863 | ;; 864 | msys) 865 | basic_machine=i686-pc 866 | os=-msys 867 | ;; 868 | mvs) 869 | basic_machine=i370-ibm 870 | os=-mvs 871 | ;; 872 | nacl) 873 | basic_machine=le32-unknown 874 | os=-nacl 875 | ;; 876 | ncr3000) 877 | basic_machine=i486-ncr 878 | os=-sysv4 879 | ;; 880 | netbsd386) 881 | basic_machine=i386-unknown 882 | os=-netbsd 883 | ;; 884 | netwinder) 885 | basic_machine=armv4l-rebel 886 | os=-linux 887 | ;; 888 | news | news700 | news800 | news900) 889 | basic_machine=m68k-sony 890 | os=-newsos 891 | ;; 892 | news1000) 893 | basic_machine=m68030-sony 894 | os=-newsos 895 | ;; 896 | news-3600 | risc-news) 897 | basic_machine=mips-sony 898 | os=-newsos 899 | ;; 900 | necv70) 901 | basic_machine=v70-nec 902 | os=-sysv 903 | ;; 904 | next | m*-next) 905 | basic_machine=m68k-next 906 | case $os in 907 | -nextstep* ) 908 | ;; 909 | -ns2*) 910 | os=-nextstep2 911 | ;; 912 | *) 913 | os=-nextstep3 914 | ;; 915 | esac 916 | ;; 917 | nh3000) 918 | basic_machine=m68k-harris 919 | os=-cxux 920 | ;; 921 | nh[45]000) 922 | basic_machine=m88k-harris 923 | os=-cxux 924 | ;; 925 | nindy960) 926 | basic_machine=i960-intel 927 | os=-nindy 928 | ;; 929 | mon960) 930 | basic_machine=i960-intel 931 | os=-mon960 932 | ;; 933 | nonstopux) 934 | basic_machine=mips-compaq 935 | os=-nonstopux 936 | ;; 937 | np1) 938 | basic_machine=np1-gould 939 | ;; 940 | neo-tandem) 941 | basic_machine=neo-tandem 942 | ;; 943 | nse-tandem) 944 | basic_machine=nse-tandem 945 | ;; 946 | nsr-tandem) 947 | basic_machine=nsr-tandem 948 | ;; 949 | nsx-tandem) 950 | basic_machine=nsx-tandem 951 | ;; 952 | op50n-* | op60c-*) 953 | basic_machine=hppa1.1-oki 954 | os=-proelf 955 | ;; 956 | openrisc | openrisc-*) 957 | basic_machine=or32-unknown 958 | ;; 959 | os400) 960 | basic_machine=powerpc-ibm 961 | os=-os400 962 | ;; 963 | OSE68000 | ose68000) 964 | basic_machine=m68000-ericsson 965 | os=-ose 966 | ;; 967 | os68k) 968 | basic_machine=m68k-none 969 | os=-os68k 970 | ;; 971 | pa-hitachi) 972 | basic_machine=hppa1.1-hitachi 973 | os=-hiuxwe2 974 | ;; 975 | paragon) 976 | basic_machine=i860-intel 977 | os=-osf 978 | ;; 979 | parisc) 980 | basic_machine=hppa-unknown 981 | os=-linux 982 | ;; 983 | parisc-*) 984 | basic_machine=hppa-`echo $basic_machine | sed 's/^[^-]*-//'` 985 | os=-linux 986 | ;; 987 | pbd) 988 | basic_machine=sparc-tti 989 | ;; 990 | pbb) 991 | basic_machine=m68k-tti 992 | ;; 993 | pc532 | pc532-*) 994 | basic_machine=ns32k-pc532 995 | ;; 996 | pc98) 997 | basic_machine=i386-pc 998 | ;; 999 | pc98-*) 1000 | basic_machine=i386-`echo $basic_machine | sed 's/^[^-]*-//'` 1001 | ;; 1002 | pentium | p5 | k5 | k6 | nexgen | viac3) 1003 | basic_machine=i586-pc 1004 | ;; 1005 | pentiumpro | p6 | 6x86 | athlon | athlon_*) 1006 | basic_machine=i686-pc 1007 | ;; 1008 | pentiumii | pentium2 | pentiumiii | pentium3) 1009 | basic_machine=i686-pc 1010 | ;; 1011 | pentium4) 1012 | basic_machine=i786-pc 1013 | ;; 1014 | pentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*) 1015 | basic_machine=i586-`echo $basic_machine | sed 's/^[^-]*-//'` 1016 | ;; 1017 | pentiumpro-* | p6-* | 6x86-* | athlon-*) 1018 | basic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'` 1019 | ;; 1020 | pentiumii-* | pentium2-* | pentiumiii-* | pentium3-*) 1021 | basic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'` 1022 | ;; 1023 | pentium4-*) 1024 | basic_machine=i786-`echo $basic_machine | sed 's/^[^-]*-//'` 1025 | ;; 1026 | pn) 1027 | basic_machine=pn-gould 1028 | ;; 1029 | power) basic_machine=power-ibm 1030 | ;; 1031 | ppc | ppcbe) basic_machine=powerpc-unknown 1032 | ;; 1033 | ppc-* | ppcbe-*) 1034 | basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'` 1035 | ;; 1036 | ppcle | powerpclittle) 1037 | basic_machine=powerpcle-unknown 1038 | ;; 1039 | ppcle-* | powerpclittle-*) 1040 | basic_machine=powerpcle-`echo $basic_machine | sed 's/^[^-]*-//'` 1041 | ;; 1042 | ppc64) basic_machine=powerpc64-unknown 1043 | ;; 1044 | ppc64-*) basic_machine=powerpc64-`echo $basic_machine | sed 's/^[^-]*-//'` 1045 | ;; 1046 | ppc64le | powerpc64little) 1047 | basic_machine=powerpc64le-unknown 1048 | ;; 1049 | ppc64le-* | powerpc64little-*) 1050 | basic_machine=powerpc64le-`echo $basic_machine | sed 's/^[^-]*-//'` 1051 | ;; 1052 | ps2) 1053 | basic_machine=i386-ibm 1054 | ;; 1055 | pw32) 1056 | basic_machine=i586-unknown 1057 | os=-pw32 1058 | ;; 1059 | rdos | rdos64) 1060 | basic_machine=x86_64-pc 1061 | os=-rdos 1062 | ;; 1063 | rdos32) 1064 | basic_machine=i386-pc 1065 | os=-rdos 1066 | ;; 1067 | rom68k) 1068 | basic_machine=m68k-rom68k 1069 | os=-coff 1070 | ;; 1071 | rm[46]00) 1072 | basic_machine=mips-siemens 1073 | ;; 1074 | rtpc | rtpc-*) 1075 | basic_machine=romp-ibm 1076 | ;; 1077 | s390 | s390-*) 1078 | basic_machine=s390-ibm 1079 | ;; 1080 | s390x | s390x-*) 1081 | basic_machine=s390x-ibm 1082 | ;; 1083 | sa29200) 1084 | basic_machine=a29k-amd 1085 | os=-udi 1086 | ;; 1087 | sb1) 1088 | basic_machine=mipsisa64sb1-unknown 1089 | ;; 1090 | sb1el) 1091 | basic_machine=mipsisa64sb1el-unknown 1092 | ;; 1093 | sde) 1094 | basic_machine=mipsisa32-sde 1095 | os=-elf 1096 | ;; 1097 | sei) 1098 | basic_machine=mips-sei 1099 | os=-seiux 1100 | ;; 1101 | sequent) 1102 | basic_machine=i386-sequent 1103 | ;; 1104 | sh) 1105 | basic_machine=sh-hitachi 1106 | os=-hms 1107 | ;; 1108 | sh5el) 1109 | basic_machine=sh5le-unknown 1110 | ;; 1111 | sh64) 1112 | basic_machine=sh64-unknown 1113 | ;; 1114 | sparclite-wrs | simso-wrs) 1115 | basic_machine=sparclite-wrs 1116 | os=-vxworks 1117 | ;; 1118 | sps7) 1119 | basic_machine=m68k-bull 1120 | os=-sysv2 1121 | ;; 1122 | spur) 1123 | basic_machine=spur-unknown 1124 | ;; 1125 | st2000) 1126 | basic_machine=m68k-tandem 1127 | ;; 1128 | stratus) 1129 | basic_machine=i860-stratus 1130 | os=-sysv4 1131 | ;; 1132 | strongarm-* | thumb-*) 1133 | basic_machine=arm-`echo $basic_machine | sed 's/^[^-]*-//'` 1134 | ;; 1135 | sun2) 1136 | basic_machine=m68000-sun 1137 | ;; 1138 | sun2os3) 1139 | basic_machine=m68000-sun 1140 | os=-sunos3 1141 | ;; 1142 | sun2os4) 1143 | basic_machine=m68000-sun 1144 | os=-sunos4 1145 | ;; 1146 | sun3os3) 1147 | basic_machine=m68k-sun 1148 | os=-sunos3 1149 | ;; 1150 | sun3os4) 1151 | basic_machine=m68k-sun 1152 | os=-sunos4 1153 | ;; 1154 | sun4os3) 1155 | basic_machine=sparc-sun 1156 | os=-sunos3 1157 | ;; 1158 | sun4os4) 1159 | basic_machine=sparc-sun 1160 | os=-sunos4 1161 | ;; 1162 | sun4sol2) 1163 | basic_machine=sparc-sun 1164 | os=-solaris2 1165 | ;; 1166 | sun3 | sun3-*) 1167 | basic_machine=m68k-sun 1168 | ;; 1169 | sun4) 1170 | basic_machine=sparc-sun 1171 | ;; 1172 | sun386 | sun386i | roadrunner) 1173 | basic_machine=i386-sun 1174 | ;; 1175 | sv1) 1176 | basic_machine=sv1-cray 1177 | os=-unicos 1178 | ;; 1179 | symmetry) 1180 | basic_machine=i386-sequent 1181 | os=-dynix 1182 | ;; 1183 | t3e) 1184 | basic_machine=alphaev5-cray 1185 | os=-unicos 1186 | ;; 1187 | t90) 1188 | basic_machine=t90-cray 1189 | os=-unicos 1190 | ;; 1191 | tile*) 1192 | basic_machine=$basic_machine-unknown 1193 | os=-linux-gnu 1194 | ;; 1195 | tx39) 1196 | basic_machine=mipstx39-unknown 1197 | ;; 1198 | tx39el) 1199 | basic_machine=mipstx39el-unknown 1200 | ;; 1201 | toad1) 1202 | basic_machine=pdp10-xkl 1203 | os=-tops20 1204 | ;; 1205 | tower | tower-32) 1206 | basic_machine=m68k-ncr 1207 | ;; 1208 | tpf) 1209 | basic_machine=s390x-ibm 1210 | os=-tpf 1211 | ;; 1212 | udi29k) 1213 | basic_machine=a29k-amd 1214 | os=-udi 1215 | ;; 1216 | ultra3) 1217 | basic_machine=a29k-nyu 1218 | os=-sym1 1219 | ;; 1220 | v810 | necv810) 1221 | basic_machine=v810-nec 1222 | os=-none 1223 | ;; 1224 | vaxv) 1225 | basic_machine=vax-dec 1226 | os=-sysv 1227 | ;; 1228 | vms) 1229 | basic_machine=vax-dec 1230 | os=-vms 1231 | ;; 1232 | vpp*|vx|vx-*) 1233 | basic_machine=f301-fujitsu 1234 | ;; 1235 | vxworks960) 1236 | basic_machine=i960-wrs 1237 | os=-vxworks 1238 | ;; 1239 | vxworks68) 1240 | basic_machine=m68k-wrs 1241 | os=-vxworks 1242 | ;; 1243 | vxworks29k) 1244 | basic_machine=a29k-wrs 1245 | os=-vxworks 1246 | ;; 1247 | wasm32) 1248 | basic_machine=wasm32-unknown 1249 | ;; 1250 | w65*) 1251 | basic_machine=w65-wdc 1252 | os=-none 1253 | ;; 1254 | w89k-*) 1255 | basic_machine=hppa1.1-winbond 1256 | os=-proelf 1257 | ;; 1258 | x64) 1259 | basic_machine=x86_64-pc 1260 | ;; 1261 | xbox) 1262 | basic_machine=i686-pc 1263 | os=-mingw32 1264 | ;; 1265 | xps | xps100) 1266 | basic_machine=xps100-honeywell 1267 | ;; 1268 | xscale-* | xscalee[bl]-*) 1269 | basic_machine=`echo $basic_machine | sed 's/^xscale/arm/'` 1270 | ;; 1271 | ymp) 1272 | basic_machine=ymp-cray 1273 | os=-unicos 1274 | ;; 1275 | z8k-*-coff) 1276 | basic_machine=z8k-unknown 1277 | os=-sim 1278 | ;; 1279 | z80-*-coff) 1280 | basic_machine=z80-unknown 1281 | os=-sim 1282 | ;; 1283 | none) 1284 | basic_machine=none-none 1285 | os=-none 1286 | ;; 1287 | 1288 | # Here we handle the default manufacturer of certain CPU types. It is in 1289 | # some cases the only manufacturer, in others, it is the most popular. 1290 | w89k) 1291 | basic_machine=hppa1.1-winbond 1292 | ;; 1293 | op50n) 1294 | basic_machine=hppa1.1-oki 1295 | ;; 1296 | op60c) 1297 | basic_machine=hppa1.1-oki 1298 | ;; 1299 | romp) 1300 | basic_machine=romp-ibm 1301 | ;; 1302 | mmix) 1303 | basic_machine=mmix-knuth 1304 | ;; 1305 | rs6000) 1306 | basic_machine=rs6000-ibm 1307 | ;; 1308 | vax) 1309 | basic_machine=vax-dec 1310 | ;; 1311 | pdp10) 1312 | # there are many clones, so DEC is not a safe bet 1313 | basic_machine=pdp10-unknown 1314 | ;; 1315 | pdp11) 1316 | basic_machine=pdp11-dec 1317 | ;; 1318 | we32k) 1319 | basic_machine=we32k-att 1320 | ;; 1321 | sh[1234] | sh[24]a | sh[24]aeb | sh[34]eb | sh[1234]le | sh[23]ele) 1322 | basic_machine=sh-unknown 1323 | ;; 1324 | sparc | sparcv8 | sparcv9 | sparcv9b | sparcv9v) 1325 | basic_machine=sparc-sun 1326 | ;; 1327 | cydra) 1328 | basic_machine=cydra-cydrome 1329 | ;; 1330 | orion) 1331 | basic_machine=orion-highlevel 1332 | ;; 1333 | orion105) 1334 | basic_machine=clipper-highlevel 1335 | ;; 1336 | mac | mpw | mac-mpw) 1337 | basic_machine=m68k-apple 1338 | ;; 1339 | pmac | pmac-mpw) 1340 | basic_machine=powerpc-apple 1341 | ;; 1342 | *-unknown) 1343 | # Make sure to match an already-canonicalized machine name. 1344 | ;; 1345 | *) 1346 | echo Invalid configuration \`$1\': machine \`$basic_machine\' not recognized 1>&2 1347 | exit 1 1348 | ;; 1349 | esac 1350 | 1351 | # Here we canonicalize certain aliases for manufacturers. 1352 | case $basic_machine in 1353 | *-digital*) 1354 | basic_machine=`echo $basic_machine | sed 's/digital.*/dec/'` 1355 | ;; 1356 | *-commodore*) 1357 | basic_machine=`echo $basic_machine | sed 's/commodore.*/cbm/'` 1358 | ;; 1359 | *) 1360 | ;; 1361 | esac 1362 | 1363 | # Decode manufacturer-specific aliases for certain operating systems. 1364 | 1365 | if [ x"$os" != x"" ] 1366 | then 1367 | case $os in 1368 | # First match some system type aliases that might get confused 1369 | # with valid system types. 1370 | # -solaris* is a basic system type, with this one exception. 1371 | -auroraux) 1372 | os=-auroraux 1373 | ;; 1374 | -solaris1 | -solaris1.*) 1375 | os=`echo $os | sed -e 's|solaris1|sunos4|'` 1376 | ;; 1377 | -solaris) 1378 | os=-solaris2 1379 | ;; 1380 | -svr4*) 1381 | os=-sysv4 1382 | ;; 1383 | -unixware*) 1384 | os=-sysv4.2uw 1385 | ;; 1386 | -gnu/linux*) 1387 | os=`echo $os | sed -e 's|gnu/linux|linux-gnu|'` 1388 | ;; 1389 | # Now accept the basic system types. 1390 | # The portable systems comes first. 1391 | # Each alternative MUST end in a * to match a version number. 1392 | # -sysv* is not here because it comes later, after sysvr4. 1393 | -gnu* | -bsd* | -mach* | -minix* | -genix* | -ultrix* | -irix* \ 1394 | | -*vms* | -sco* | -esix* | -isc* | -aix* | -cnk* | -sunos | -sunos[34]*\ 1395 | | -hpux* | -unos* | -osf* | -luna* | -dgux* | -auroraux* | -solaris* \ 1396 | | -sym* | -kopensolaris* | -plan9* \ 1397 | | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \ 1398 | | -aos* | -aros* | -cloudabi* | -sortix* \ 1399 | | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \ 1400 | | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \ 1401 | | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* \ 1402 | | -bitrig* | -openbsd* | -solidbsd* | -libertybsd* \ 1403 | | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \ 1404 | | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \ 1405 | | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \ 1406 | | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* \ 1407 | | -chorusos* | -chorusrdb* | -cegcc* | -glidix* \ 1408 | | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \ 1409 | | -midipix* | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \ 1410 | | -linux-newlib* | -linux-musl* | -linux-uclibc* \ 1411 | | -uxpv* | -beos* | -mpeix* | -udk* | -moxiebox* \ 1412 | | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \ 1413 | | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \ 1414 | | -storm-chaos* | -tops10* | -tenex* | -tops20* | -its* \ 1415 | | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \ 1416 | | -morphos* | -superux* | -rtmk* | -rtmk-nova* | -windiss* \ 1417 | | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \ 1418 | | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -es* \ 1419 | | -onefs* | -tirtos* | -phoenix* | -fuchsia* | -redox*) 1420 | # Remember, each alternative MUST END IN *, to match a version number. 1421 | ;; 1422 | -qnx*) 1423 | case $basic_machine in 1424 | x86-* | i*86-*) 1425 | ;; 1426 | *) 1427 | os=-nto$os 1428 | ;; 1429 | esac 1430 | ;; 1431 | -nto-qnx*) 1432 | ;; 1433 | -nto*) 1434 | os=`echo $os | sed -e 's|nto|nto-qnx|'` 1435 | ;; 1436 | -sim | -es1800* | -hms* | -xray | -os68k* | -none* | -v88r* \ 1437 | | -windows* | -osx | -abug | -netware* | -os9* | -beos* | -haiku* \ 1438 | | -macos* | -mpw* | -magic* | -mmixware* | -mon960* | -lnews*) 1439 | ;; 1440 | -mac*) 1441 | os=`echo $os | sed -e 's|mac|macos|'` 1442 | ;; 1443 | -linux-dietlibc) 1444 | os=-linux-dietlibc 1445 | ;; 1446 | -linux*) 1447 | os=`echo $os | sed -e 's|linux|linux-gnu|'` 1448 | ;; 1449 | -sunos5*) 1450 | os=`echo $os | sed -e 's|sunos5|solaris2|'` 1451 | ;; 1452 | -sunos6*) 1453 | os=`echo $os | sed -e 's|sunos6|solaris3|'` 1454 | ;; 1455 | -opened*) 1456 | os=-openedition 1457 | ;; 1458 | -os400*) 1459 | os=-os400 1460 | ;; 1461 | -wince*) 1462 | os=-wince 1463 | ;; 1464 | -osfrose*) 1465 | os=-osfrose 1466 | ;; 1467 | -osf*) 1468 | os=-osf 1469 | ;; 1470 | -utek*) 1471 | os=-bsd 1472 | ;; 1473 | -dynix*) 1474 | os=-bsd 1475 | ;; 1476 | -acis*) 1477 | os=-aos 1478 | ;; 1479 | -atheos*) 1480 | os=-atheos 1481 | ;; 1482 | -syllable*) 1483 | os=-syllable 1484 | ;; 1485 | -386bsd) 1486 | os=-bsd 1487 | ;; 1488 | -ctix* | -uts*) 1489 | os=-sysv 1490 | ;; 1491 | -nova*) 1492 | os=-rtmk-nova 1493 | ;; 1494 | -ns2) 1495 | os=-nextstep2 1496 | ;; 1497 | -nsk*) 1498 | os=-nsk 1499 | ;; 1500 | # Preserve the version number of sinix5. 1501 | -sinix5.*) 1502 | os=`echo $os | sed -e 's|sinix|sysv|'` 1503 | ;; 1504 | -sinix*) 1505 | os=-sysv4 1506 | ;; 1507 | -tpf*) 1508 | os=-tpf 1509 | ;; 1510 | -triton*) 1511 | os=-sysv3 1512 | ;; 1513 | -oss*) 1514 | os=-sysv3 1515 | ;; 1516 | -svr4) 1517 | os=-sysv4 1518 | ;; 1519 | -svr3) 1520 | os=-sysv3 1521 | ;; 1522 | -sysvr4) 1523 | os=-sysv4 1524 | ;; 1525 | # This must come after -sysvr4. 1526 | -sysv*) 1527 | ;; 1528 | -ose*) 1529 | os=-ose 1530 | ;; 1531 | -es1800*) 1532 | os=-ose 1533 | ;; 1534 | -xenix) 1535 | os=-xenix 1536 | ;; 1537 | -*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*) 1538 | os=-mint 1539 | ;; 1540 | -aros*) 1541 | os=-aros 1542 | ;; 1543 | -zvmoe) 1544 | os=-zvmoe 1545 | ;; 1546 | -dicos*) 1547 | os=-dicos 1548 | ;; 1549 | -pikeos*) 1550 | # Until real need of OS specific support for 1551 | # particular features comes up, bare metal 1552 | # configurations are quite functional. 1553 | case $basic_machine in 1554 | arm*) 1555 | os=-eabi 1556 | ;; 1557 | *) 1558 | os=-elf 1559 | ;; 1560 | esac 1561 | ;; 1562 | -nacl*) 1563 | ;; 1564 | -ios) 1565 | ;; 1566 | -none) 1567 | ;; 1568 | *) 1569 | # Get rid of the `-' at the beginning of $os. 1570 | os=`echo $os | sed 's/[^-]*-//'` 1571 | echo Invalid configuration \`$1\': system \`$os\' not recognized 1>&2 1572 | exit 1 1573 | ;; 1574 | esac 1575 | else 1576 | 1577 | # Here we handle the default operating systems that come with various machines. 1578 | # The value should be what the vendor currently ships out the door with their 1579 | # machine or put another way, the most popular os provided with the machine. 1580 | 1581 | # Note that if you're going to try to match "-MANUFACTURER" here (say, 1582 | # "-sun"), then you have to tell the case statement up towards the top 1583 | # that MANUFACTURER isn't an operating system. Otherwise, code above 1584 | # will signal an error saying that MANUFACTURER isn't an operating 1585 | # system, and we'll never get to this point. 1586 | 1587 | case $basic_machine in 1588 | score-*) 1589 | os=-elf 1590 | ;; 1591 | spu-*) 1592 | os=-elf 1593 | ;; 1594 | *-acorn) 1595 | os=-riscix1.2 1596 | ;; 1597 | arm*-rebel) 1598 | os=-linux 1599 | ;; 1600 | arm*-semi) 1601 | os=-aout 1602 | ;; 1603 | c4x-* | tic4x-*) 1604 | os=-coff 1605 | ;; 1606 | c8051-*) 1607 | os=-elf 1608 | ;; 1609 | hexagon-*) 1610 | os=-elf 1611 | ;; 1612 | tic54x-*) 1613 | os=-coff 1614 | ;; 1615 | tic55x-*) 1616 | os=-coff 1617 | ;; 1618 | tic6x-*) 1619 | os=-coff 1620 | ;; 1621 | # This must come before the *-dec entry. 1622 | pdp10-*) 1623 | os=-tops20 1624 | ;; 1625 | pdp11-*) 1626 | os=-none 1627 | ;; 1628 | *-dec | vax-*) 1629 | os=-ultrix4.2 1630 | ;; 1631 | m68*-apollo) 1632 | os=-domain 1633 | ;; 1634 | i386-sun) 1635 | os=-sunos4.0.2 1636 | ;; 1637 | m68000-sun) 1638 | os=-sunos3 1639 | ;; 1640 | m68*-cisco) 1641 | os=-aout 1642 | ;; 1643 | mep-*) 1644 | os=-elf 1645 | ;; 1646 | mips*-cisco) 1647 | os=-elf 1648 | ;; 1649 | mips*-*) 1650 | os=-elf 1651 | ;; 1652 | or32-*) 1653 | os=-coff 1654 | ;; 1655 | *-tti) # must be before sparc entry or we get the wrong os. 1656 | os=-sysv3 1657 | ;; 1658 | sparc-* | *-sun) 1659 | os=-sunos4.1.1 1660 | ;; 1661 | pru-*) 1662 | os=-elf 1663 | ;; 1664 | *-be) 1665 | os=-beos 1666 | ;; 1667 | *-haiku) 1668 | os=-haiku 1669 | ;; 1670 | *-ibm) 1671 | os=-aix 1672 | ;; 1673 | *-knuth) 1674 | os=-mmixware 1675 | ;; 1676 | *-wec) 1677 | os=-proelf 1678 | ;; 1679 | *-winbond) 1680 | os=-proelf 1681 | ;; 1682 | *-oki) 1683 | os=-proelf 1684 | ;; 1685 | *-hp) 1686 | os=-hpux 1687 | ;; 1688 | *-hitachi) 1689 | os=-hiux 1690 | ;; 1691 | i860-* | *-att | *-ncr | *-altos | *-motorola | *-convergent) 1692 | os=-sysv 1693 | ;; 1694 | *-cbm) 1695 | os=-amigaos 1696 | ;; 1697 | *-dg) 1698 | os=-dgux 1699 | ;; 1700 | *-dolphin) 1701 | os=-sysv3 1702 | ;; 1703 | m68k-ccur) 1704 | os=-rtu 1705 | ;; 1706 | m88k-omron*) 1707 | os=-luna 1708 | ;; 1709 | *-next) 1710 | os=-nextstep 1711 | ;; 1712 | *-sequent) 1713 | os=-ptx 1714 | ;; 1715 | *-crds) 1716 | os=-unos 1717 | ;; 1718 | *-ns) 1719 | os=-genix 1720 | ;; 1721 | i370-*) 1722 | os=-mvs 1723 | ;; 1724 | *-next) 1725 | os=-nextstep3 1726 | ;; 1727 | *-gould) 1728 | os=-sysv 1729 | ;; 1730 | *-highlevel) 1731 | os=-bsd 1732 | ;; 1733 | *-encore) 1734 | os=-bsd 1735 | ;; 1736 | *-sgi) 1737 | os=-irix 1738 | ;; 1739 | *-siemens) 1740 | os=-sysv4 1741 | ;; 1742 | *-masscomp) 1743 | os=-rtu 1744 | ;; 1745 | f30[01]-fujitsu | f700-fujitsu) 1746 | os=-uxpv 1747 | ;; 1748 | *-rom68k) 1749 | os=-coff 1750 | ;; 1751 | *-*bug) 1752 | os=-coff 1753 | ;; 1754 | *-apple) 1755 | os=-macos 1756 | ;; 1757 | *-atari*) 1758 | os=-mint 1759 | ;; 1760 | *) 1761 | os=-none 1762 | ;; 1763 | esac 1764 | fi 1765 | 1766 | # Here we handle the case where we know the os, and the CPU type, but not the 1767 | # manufacturer. We pick the logical manufacturer. 1768 | vendor=unknown 1769 | case $basic_machine in 1770 | *-unknown) 1771 | case $os in 1772 | -riscix*) 1773 | vendor=acorn 1774 | ;; 1775 | -sunos*) 1776 | vendor=sun 1777 | ;; 1778 | -cnk*|-aix*) 1779 | vendor=ibm 1780 | ;; 1781 | -beos*) 1782 | vendor=be 1783 | ;; 1784 | -hpux*) 1785 | vendor=hp 1786 | ;; 1787 | -mpeix*) 1788 | vendor=hp 1789 | ;; 1790 | -hiux*) 1791 | vendor=hitachi 1792 | ;; 1793 | -unos*) 1794 | vendor=crds 1795 | ;; 1796 | -dgux*) 1797 | vendor=dg 1798 | ;; 1799 | -luna*) 1800 | vendor=omron 1801 | ;; 1802 | -genix*) 1803 | vendor=ns 1804 | ;; 1805 | -mvs* | -opened*) 1806 | vendor=ibm 1807 | ;; 1808 | -os400*) 1809 | vendor=ibm 1810 | ;; 1811 | -ptx*) 1812 | vendor=sequent 1813 | ;; 1814 | -tpf*) 1815 | vendor=ibm 1816 | ;; 1817 | -vxsim* | -vxworks* | -windiss*) 1818 | vendor=wrs 1819 | ;; 1820 | -aux*) 1821 | vendor=apple 1822 | ;; 1823 | -hms*) 1824 | vendor=hitachi 1825 | ;; 1826 | -mpw* | -macos*) 1827 | vendor=apple 1828 | ;; 1829 | -*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*) 1830 | vendor=atari 1831 | ;; 1832 | -vos*) 1833 | vendor=stratus 1834 | ;; 1835 | esac 1836 | basic_machine=`echo $basic_machine | sed "s/unknown/$vendor/"` 1837 | ;; 1838 | esac 1839 | 1840 | echo $basic_machine$os 1841 | exit 1842 | 1843 | # Local variables: 1844 | # eval: (add-hook 'write-file-functions 'time-stamp) 1845 | # time-stamp-start: "timestamp='" 1846 | # time-stamp-format: "%:y-%02m-%02d" 1847 | # time-stamp-end: "'" 1848 | # End: 1849 | --------------------------------------------------------------------------------