├── .gitignore ├── Changes ├── Changes-3.0.md ├── Changes-4.0.md ├── Changes-4.1.md ├── LICENSE ├── MANIFEST.SKIP ├── Starterpage.md ├── devel ├── installtapper └── tapper-github.gmf ├── dist.ini ├── examples ├── README ├── dpath │ ├── .gitignore │ ├── CTCS_ratio.gnuplot │ ├── KernBench_ratio.gnuplot │ ├── Tapper_ratio.gnuplot │ ├── benchmark_perlformance_data.mas │ ├── between.mas │ ├── context_meta_info.mas │ ├── debug_api.mas │ ├── download-reports.pl │ ├── get_known_good_xen_unstable.mas │ ├── get_precondition.mas │ ├── groupstats.mas │ ├── hello.mas │ ├── hello.tt │ ├── limit_by_date.mas │ ├── metadata_recovery.py │ ├── metareports │ │ ├── ctcs_summary_failed_tests.gnuplot │ │ ├── tapper_rotation_overview_1day.gnuplot │ │ ├── tapper_rotation_overview_1month.gnuplot │ │ ├── tapper_rotation_overview_1week.gnuplot │ │ ├── tapper_rotation_overview_alltime.gnuplot │ │ ├── tapper_tests_per_week.gnuplot │ │ ├── tapper_topic_ratio.gnuplot │ │ └── teaser.gnuplot │ ├── phoronix.py │ ├── plain.mas │ ├── proc-interrupts.mas │ ├── python_mason.py │ ├── report_dumper.mas │ ├── slbench_evaluate_language_series.pl │ ├── suite-percentage.tt │ ├── table1.mas │ ├── table2.mas │ ├── table_cs_vs_platform.py │ ├── table_cs_vs_workload.py │ ├── tapper-cmd.mas │ ├── tapper-reports-receiver.mas │ ├── tapper-schema.mas │ ├── tapper.mas │ ├── warmup.gnuplot │ ├── warmup1.gnuplot │ ├── warmup2_json.mas │ ├── warmup3_slbench.mas │ ├── warmup4_kvmpassthrough.mas │ └── xen_weekly.py ├── how_things_work_together │ ├── README │ ├── bigfish.mpc │ ├── bigfish.sh │ ├── bigfish_autoreport.sh │ ├── bigfish_simple.sh │ └── report.mas ├── preconditions │ ├── autoinstall.yml │ ├── interdep.sc │ ├── kernel_boot.mpc │ ├── kernel_boot_simple.mpc │ ├── kernel_build.mpc │ ├── kernel_reboot.mpc │ ├── kvm.yml │ ├── kvm_autoinstall.yml │ ├── notification.yml │ ├── oprofile.mpc │ ├── precondition_for_discover.yml │ ├── simulator.yml │ ├── suspend_resume.yml │ ├── test_env.mpc │ ├── testplan.mpc │ ├── xen_temare.yml │ └── xen_windows.yml └── testplans │ ├── include │ ├── defaultbenchmarks │ ├── defaults │ └── distrodetails │ ├── kernel │ ├── benchmark │ └── generic │ │ ├── benchmark │ │ └── test │ └── topic │ └── xen │ └── generic │ ├── guest-start-template.sh │ ├── guest-template.svm │ ├── test │ └── upload-xen-dmesg.sh └── lib └── Tapper.pm /.gitignore: -------------------------------------------------------------------------------- 1 | .* 2 | !.gitignore 3 | Tapper-* 4 | Debian_CPANTS.txt 5 | -------------------------------------------------------------------------------- /Changes: -------------------------------------------------------------------------------- 1 | Revision history for {{$dist->name}} 2 | 3 | {{$NEXT}} 4 | - Complete benchmarking sub system, consisting of 5 | * key/value schema to describe data points, usually for benchmark results 6 | * corresponding backend store, scaling to millions of entries 7 | * dedicated query api to fetch data points by key/values 8 | * native charting in Web UI (to replace 100% of codespeed and typical gnuplot uses) 9 | * REST-like data export of those data 10 | - Scale-up 11 | * to millions of testruns, reports, and data points 12 | * thousands of testruns running at the same time 13 | * dedicated MySQL optimizations 14 | - Pool hosts 15 | * hide multiple "anonymous" machines behind a single machine name 16 | * define quota how many of such hosts of a pool can run at same time 17 | * think of a cloud of instances 18 | - Command-line interface harmonized 19 | * less scattered tools 20 | * just one single frontend command "tapper" 21 | - Testplan improvements 22 | * merged feature set with "interdependent scenarios" 23 | * better preview/debugging support 24 | * REST api to submit tests and testplans 25 | - Web UI improvements 26 | * modernized (yes, more Javascript) 27 | * optimizations for speed and traffic 28 | * more powerful navigation and filters 29 | * UI to maintain continuous testrun rotation (pause, continue, edit, delete, clone) 30 | - Tapper-autoreport/bash-test-utils 31 | * better separation from Tapper via plugins/hooks 32 | * support Tapper benchmarking 33 | - General refactoring 34 | * Hacking - raw SQL support for optional performance tweaks 35 | * more plugin'ification, to separate internal systems 36 | * merged the two databases, which makes the automation layer better integrated 37 | * database connection re-use (allowing thousands of testruns at the same time) 38 | 39 | 4.1.1 2012-11-19 40 | - rerelease for CPAN ID TAPPER 41 | - new command 'tapper init': 42 | creates ~/.tapper/ with defaults and examples 43 | for an easier start as normal user 44 | 45 | 4.1 2012-10-23 46 | * Databases 47 | - refactoring to also work with PostgreSQL 48 | 49 | * Automation 50 | - scheduler 10x speedup 51 | - first Cobbler support 52 | - host blacklisting per queue 53 | 54 | * Tools 55 | - CLI consolidation/harmonization 56 | 57 | * Documentation 58 | - migrated to POD for maintainability 59 | - complex precondition and testplan examples 60 | 61 | * Misc 62 | - overall cleanup in sync with upstream upgrades 63 | * autotest 0.14.x 64 | * Perl 5.16 65 | * Catalyst 5.9 66 | 67 | 4.0 2012-05-16 68 | * Automation 69 | - better linux32 chroot/exec support 70 | - persistent automation layer, based on event-queues 71 | (Tapper::MCP, Tapper::MCP::MessageReceiver) 72 | - introduce SSH-connect to test without complete machine setup 73 | - support suspend/resume testing; 74 | via abstract central 'actions' to be called from remote clients 75 | (Tapper::Action) 76 | - conditionally trigger notifications on incoming results 77 | (Tapper::Notification) 78 | - keep-alive mechanics for broken hosts 79 | - much better scriptability everywhere to support strangest 80 | requirements 81 | - reworked TaskJuggler/Testplan bridge 82 | (Tapper::Testplan) 83 | 84 | * Testsuites 85 | - Tapper-autotest wrapper: 86 | - now send+upload virtually all result details+files 87 | - allow use snapshots to not suffer from upstream changes 88 | - Tapper-autoreport: better virtualization support (probably the 89 | world's current best Xen/KVM host/guest detection heuristics, 90 | really) 91 | - better Perl::Formance benchmarking integration 92 | 93 | * Reports database 94 | - store attachments bzip2 compressed (optionally compress already 95 | existing attachments) 96 | 97 | * Reports API 98 | - allow passthrough of incoming results to 3rd party applications 99 | (e.g. extract benchmark results and pass them along them to external 100 | graph rendering - aka. "level 2 receivers") 101 | - more robust TAP::Archive support 102 | 103 | * Query API 104 | - easier attachment downloading 105 | - QueryAPI now available in testplan template to allows generate 106 | testruns based on older results (think of "use last successful 107 | aka. known-good Xen changeset for another complicated test") 108 | 109 | * Web GUI 110 | - user authentication (via PAM) 111 | - reworked filter framework 112 | - better show current scheduling state 113 | - configurable site customizations for non-OSRC instances 114 | 115 | * CLI 116 | - new frontend tool 'tapper' 117 | - more subcommands for user/notification/testplan handling 118 | - testplan development support 119 | 120 | * misc 121 | - better configurable control over grub entry writing 122 | - utils to fake/test the automation layer (Tapper::Fake) 123 | 124 | * hack on Tapper 125 | - use Dist::Zilla to author Perl libs 126 | 127 | * Tapper 128 | - zero-content lib as primary entry point for Tapper info 129 | 130 | 3.0 2011-04-08 131 | - for the initial Tapper feature overview (version 3) see 132 | http://www.amd64.org/support/automated-test-infrastructure.html 133 | -------------------------------------------------------------------------------- /Changes-3.0.md: -------------------------------------------------------------------------------- 1 | ## Tapper release 3.0 2011-03-02 (first public release) 2 | 3 | Tapper is an infrastructure for all aspects of testing inclusive 4 | Operating Systems and Virtualization. 5 | 6 | ## Tapper contains 7 | 8 | * Automation 9 | * Machine Scheduling 10 | * Command line utils 11 | * Web Frontend application 12 | * Support for writing tests 13 | * Powerful result evaluation API 14 | * Testplan support with TaskJuggler 15 | * Many use-cases from tracking test results to benchmarking to full OS testing including virtualization 16 | 17 | ## Feature overview 18 | 19 | ### Automation 20 | 21 | * Network boot (PXE + TFTP + NFS) 22 | * Machine self-setup driven by abstract specfiles 23 | * Image-based or kickstart/autoyast based installation 24 | * Lightweight status API to easily plug in foreign installers 25 | * Support for virtualization setups (Xen and KVM) 26 | * Inject packages, programs, files into hosts/guests 27 | * Control execution of test scripts in hosts/guests 28 | * Interdependent setup and sync of co-operating machines 29 | * Complex timeout handling for complete test cycle, inclusive virtualizated guest 30 | * Reboot handling 31 | * Console logging 32 | * Hardware reset 33 | 34 | ### Scheduling 35 | 36 | * Optimize utilization of pools with "not enough" machines for "too many use-cases" 37 | * Multiplex queues of freely definable use-cases (e.g., Xen, KVM, baremetal kernels, ad-hoc tests, benchmarks) with different bandwidths 38 | * Pluggable core scheduling algorithm (default "Weighted Fair Queuing") 39 | * Feature driven host matching (by memory, cores, vendor, etc. or complex combinations) 40 | * Dynamic queue and host management (add, delete, de/activate) 41 | * Host/queue binding for dedicated scheduling 42 | * Auto re-queuing for continuous testing 43 | * Multi-host scenarios 44 | 45 | ### Web Application 46 | 47 | * Frontend to 2 databases: testruns and reports 48 | * Providing a "management view" and high-level test result evaluation 49 | * Overview lists and detailed zoom-in, GREEN/YELLOW/RED coloring 50 | * Filters over time, testsuites, machines, success status 51 | * RSS feeds on such filters 52 | * Visualize groups of connected results (virtualized neighbour guests) 53 | * Control start of testruns 54 | 55 | ### Result Evaluation 56 | 57 | * Programmable complement to the web frontend 58 | * Allow complex queries on the test result database 59 | * No client side toolchain neccessary 60 | * Easy API to send and get back queries embedded in templates 61 | * API allows SQL and XPath like queries in an abstract way 62 | * Presentation about the query interface at the YAPC::Europe 2009 63 | 64 | ### Testplan Support 65 | 66 | * Testplans combine many Tapper features into concise points of interest 67 | * Manage testplan hierarchy allowing specfile reuse for complex testplan matrix 68 | * Interoperation with TaskJuggler on dedicated QA tasks for automatic scheduling and reporting 69 | * Consequently a complete focused QA life cycle of planning, execution and reporting 70 | 71 | ### Support for writing tests 72 | 73 | * Format test results based on standard Test Anything Protocol (TAP) 74 | * Existing standard TAP toolchains available for about 20 programming languages 75 | * Dedicated Tapper support for Shell, Perl, Python available 76 | 77 | ### Many use-cases 78 | 79 | * Kernel testing (stress, function, reboot) 80 | * Virtualization Testing (Xen, KVM) 81 | * Test matrix of host/guest combinations 82 | * Distribution testing (like OS or compilers) 83 | * Multi-machine scenarios 84 | * Complex result evaluation 85 | 86 | ### Technology 87 | 88 | * Test Anything Protocol (TAP) 89 | * Core system written in Perl and CPAN 90 | * DB independent via OR mapper, developed on MySQL and SQLite 91 | * HTML/CSS, cautious Javascript 92 | * Language agnostic testing (e.g, Perl/Python/Shell test suites) 93 | * PXE, GRUB, TFTP, NFS boot automation 94 | * Strong decoupling of functional layers (webgui, testsuites, automation) to allow you amalgamate own infrastructures 95 | 96 | ## Correlation to autotest.kernel.org 97 | 98 | The main focus of autotest.kernel.org project is on testing the Linux 99 | kernel. It provides a broad coverage of kernel functionality testing 100 | and wrappers of many existing test suites. 101 | 102 | Tapper provides many complex scenarios, like virtualization (Xen/KVM), 103 | distribution testing (RHEL, SLES, Debian), SimNow testing and 104 | benchmarking. Tapper can schedule them all multiplexed with according 105 | bandwidths over large or small machine pools. The autotest.kernel.org 106 | client can be used in a Tapper infrastructure via a thin wrapper that 107 | utilizes the TAP export we provided to the autotest project. Tapper 108 | then complements it with Testplan support, a result database and a 109 | homogeneous result evaluation API. More information about Tapper: 110 | Downloading Tapper 111 | 112 | * https://github.com/amd 113 | * search.cpan.org/~amd 114 | 115 | ## Tapper Support 116 | 117 | * IRC: #tapper (irc.freenode.net) 118 | * Mailing List: tapper@amd64.org 119 | -------------------------------------------------------------------------------- /Changes-4.0.md: -------------------------------------------------------------------------------- 1 | * She's mad at everybody. She's even mad at the ice cream man. 2 | "Why does the ice cream truck have to come just before lunch 3 | or just before dinner, spoil the children's appetite?" 4 | I have to listen to that. I hear that 3 time a week, you know 5 | that's 12 times a month. 6 | 7 | Columbo, "The Most Crucial Game" (1972) 8 | 9 | ## Tapper release 4.0 codename "Columbo" 2012-05-16 10 | 11 | ### Automation 12 | 13 | * better linux32 chroot/exec support 14 | * persistent automation layer, based on event-queues 15 | (Tapper::MCP, Tapper::MCP::MessageReceiver) 16 | * introduce SSH-connect to test without complete machine setup 17 | * support suspend/resume testing; 18 | via abstract central 'actions' to be called from remote clients 19 | (Tapper::Action) 20 | * conditionally trigger notifications on incoming results 21 | (Tapper::Notification) 22 | * keep-alive mechanics for broken hosts 23 | * much better scriptability everywhere to support strangest 24 | requirements 25 | * reworked TaskJuggler/Testplan bridge 26 | (Tapper::Testplan) 27 | 28 | ### Testsuites 29 | 30 | * Tapper-autotest wrapper: 31 | * now send+upload virtually all result details+files 32 | * allow use snapshots to not suffer from upstream changes 33 | * Tapper-autoreport: better virtualization support (probably the 34 | world's current best Xen/KVM host/guest detection heuristics, 35 | really) 36 | * better Perl::Formance benchmarking integration 37 | 38 | ### Reports database 39 | 40 | * store attachments bzip2 compressed (optionally compress already 41 | existing attachments) 42 | 43 | ### Reports API 44 | 45 | * allow passthrough of incoming results to 3rd party applications 46 | (e.g. extract benchmark results and pass them along them to external 47 | graph rendering - aka. "level 2 receivers") 48 | * more robust TAP::Archive support 49 | 50 | ### Query API 51 | 52 | * easier attachment downloading 53 | * QueryAPI now available in testplan template to allows generate 54 | testruns based on older results (think of "use last successful 55 | aka. known-good Xen changeset for another complicated test") 56 | 57 | ### Web GUI 58 | 59 | * user authentication (via PAM) 60 | * reworked filter framework 61 | * better show current scheduling state 62 | * configurable site customizations for non-OSRC instances 63 | 64 | ### CLI 65 | 66 | * new frontend tool 'tapper' 67 | * more subcommands for user/notification/testplan handling 68 | * testplan development support 69 | 70 | ### misc 71 | 72 | * better configurable control over grub entry writing 73 | * utils to fake/test the automation layer (Tapper::Fake) 74 | 75 | ### Hacking on Tapper 76 | 77 | * we now use Dist::Zilla to author Perl libs 78 | * plugins: 79 | * Dist::Zilla::PluginBundle::AMD 80 | * PodWeaver::PluginBundle::AMD 81 | * Task::BeLike::AMD 82 | 83 | -------------------------------------------------------------------------------- /Changes-4.1.md: -------------------------------------------------------------------------------- 1 | * Lacey: Looks like a young Omar Sharif. 2 | Cagney: He's got crooked teeth. 3 | Lacey: You know, Christine, you're very critical. That's your trouble with men. You want them all to be perfect. 4 | Cagney: No, I just have a thing about teeth. 5 | 6 | Cagney & Lacey, "Let Them Eat Pretzels" (1983) 7 | 8 | ## Tapper release 4.1 codename "Cagney & Lacey" 2012-10-23 9 | 10 | ### Databases 11 | 12 | * refactoring to also work with PostgreSQL 13 | 14 | ### Automation 15 | 16 | * scheduler 10x speedup 17 | * host blacklisting per queue 18 | * beginning Cobbler support 19 | 20 | ### Tools 21 | 22 | * CLI command harmonization 23 | 24 | ### Documentation 25 | 26 | * migrated to POD for maintainability 27 | * complex precondition and testplan examples 28 | 29 | ### Misc 30 | 31 | * overall cleanup to sync with upstream technologies 32 | * autotest 0.14.x 33 | * Perl 5.16 34 | * Catalyst 5.9 35 | * throw away BindLex, NEXT 36 | * use other Mason view 37 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2008-2012, Advanced Micro Devices, Inc. 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are 6 | met: 7 | 8 | - Redistributions of source code must retain the above copyright 9 | notice, this list of conditions and the following disclaimer. 10 | 11 | - Redistributions in binary form must reproduce the above copyright 12 | notice, this list of conditions and the following disclaimer in the 13 | documentation and/or other materials provided with the distribution. 14 | 15 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 16 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 17 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 18 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 19 | HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 20 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 21 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 22 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 23 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 24 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 25 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | -------------------------------------------------------------------------------- /MANIFEST.SKIP: -------------------------------------------------------------------------------- 1 | examples 2 | devel 3 | -------------------------------------------------------------------------------- /Starterpage.md: -------------------------------------------------------------------------------- 1 | ### Welcome to Tapper 2 | 3 | #### Tapper is... 4 | 5 | * a test *infrastructure*. 6 | * designed to allow an easy start. 7 | * designed to extend to complex test labs 8 | * able to unify lots of different use-cases under a common umbrella 9 | * generic enough to compete with the future 10 | * mature 11 | 12 | #### Tapper originated... 13 | 14 | at AMD's [Operating System Research Center](http://developer.amd.com/tools-and-sdks/open-source/tapper/) where it 15 | serves as umbrella for all QA. It integrates 10+ years of QA know-how 16 | and Linux testing expertise since 2008. It is now also the 17 | infrastructure behind the Perl benchmarking infrastructure 18 | [Perl::Formance](http://perlformance.net). 19 | 20 | #### Tapper provides... 21 | 22 | * extreme continuous integration through setting up complete machines 23 | from scratch, either image-based or via kickstart/autoyast/preseed. 24 | * alternative easy ssh-based test invocation 25 | * test facilities for the Linux kernel, virtualization based on Xen 26 | and KVM, and running in simulators (like simnow) 27 | * support for lots of other scenarios, like suspend/resume or distributed 28 | network tests 29 | * benchmark tracking facilities 30 | * powerful, advanced query language for test result evaluation and 31 | forensics 32 | * a powerful scheduler to maximize utilization of a machine pool with 33 | different use-cases organized in bandwidth-driven queues 34 | 35 | #### Tapper APIs... 36 | 37 | * are easy, without requiring dependencies to talk to them 38 | * allow integration of your existing infrastructure with nothing more 39 | like shell scripts using "echo" and "netcat" 40 | 41 | #### Tapper encourages... 42 | 43 | the use of language diversity for writing tests by building upon TAP, 44 | the mature "Test Anything Protocol", supported in about 20 programming 45 | languages. 46 | 47 | #### Tapper co-operates... 48 | 49 | with 50 | 51 | * autotest 52 | * Codespeed 53 | * TaskJuggler 54 | * any TAP emitters 55 | 56 | #### Related material 57 | 58 | * [Overview](http://www.tapper-testing.org/docs/linuxcon_eu_2011_linux_and_virtualization_testing_with_tapper.pdf) 59 | presentation at Linuxcon Europe 2011 60 | * [TAP Juggling](http://www.tapper-testing.org/docs/yapc_eu_2011_tapjuggling.pdf) 61 | at YAPC::EU 2011 62 | * [Query API](http://www.tapper-testing.org/docs/yapc_eu_2009_cinderella_tap.pdf) 63 | at YAPC::EU 2009 64 | * [Tapper on github](http://github.com/tapper) 65 | -------------------------------------------------------------------------------- /devel/installtapper: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | # ----------------------------------------------------------------- 4 | # 5 | # This file is behind http://tapper-testing.org/installtapper to be 6 | # used as install-everything installer: 7 | # 8 | # curl -kL http://tapper-testing.org/installtapper | bash 9 | # 10 | # ----------------------------------------------------------------- 11 | 12 | if [ -e ~/.tapper ] ; then 13 | echo "You already have a ~/.tapper/ -- for updating it please use" 14 | echo " tapper init --default" 15 | echo "Exiting." 16 | exit 1 17 | fi 18 | 19 | echo "" 20 | echo "**************************************************" 21 | echo "" 22 | echo "Please install dependencies first, like this:" 23 | echo "" 24 | echo "sudo apt-get install gcc" 25 | echo "sudo apt-get install make" 26 | echo "sudo apt-get install libsqlite3-dev" 27 | echo "sudo apt-get install libexpat1-dev" 28 | echo "sudo apt-get install libxml2-dev" 29 | echo "sudo apt-get install libz-dev" 30 | echo "sudo apt-get install libgmp-dev" 31 | echo "" 32 | echo "**************************************************" 33 | echo "" 34 | sleep 3 35 | 36 | curl -kL http://install.perlbrew.pl | bash 37 | source ~/perl5/perlbrew/etc/bashrc 38 | perlbrew install --as tapper-perl perl-5.16.3 39 | perlbrew use tapper-perl 40 | 41 | curl -L http://cpanmin.us | perl - App::cpanminus 42 | cpanm --force Template::Plugin::Autoformat 43 | cpanm Catalyst::Action::RenderView 44 | cpanm Task::Tapper::Hello::World 45 | cpanm Task::Tapper::Hello::World::Automation 46 | cpanm SCHWIGON/TAP-Harness-Archive-0.15.tar.gz 47 | 48 | ( echo "y" ; echo "y" ) | tapper init --default 49 | 50 | echo "" 51 | echo "**************************************************" 52 | echo "" 53 | echo "Don't forget to add this line to your ~/.bashrc:" 54 | echo "" 55 | echo " source ~/perl5/perlbrew/etc/bashrc" 56 | echo "" 57 | echo "and setting your Perl version permanently with" 58 | echo "" 59 | echo " perlbrew switch tapper-perl" 60 | echo "" 61 | echo "**************************************************" 62 | echo "" 63 | 64 | -------------------------------------------------------------------------------- /devel/tapper-github.gmf: -------------------------------------------------------------------------------- 1 | # -*- mode: yaml -*- 2 | 3 | # GitMeta spec for Tapper repositories on github 4 | # See 5 | # http://search.cpan.org/~mschilli/GitMeta/lib/GitMeta.pm 6 | 7 | - 8 | type: Github 9 | user: tapper 10 | 11 | -------------------------------------------------------------------------------- /dist.ini: -------------------------------------------------------------------------------- 1 | name = Tapper 2 | author = AMD OSRC Tapper Team 3 | license = FreeBSD 4 | copyright_holder = Advanced Micro Devices, Inc. 5 | 6 | [@TAPPER] 7 | dist = Tapper 8 | repository_at = github 9 | 10 | [ManifestSkip] 11 | -------------------------------------------------------------------------------- /examples/README: -------------------------------------------------------------------------------- 1 | This subdir contains a collection of examples 2 | for various subsystems of Tapper: query api templates, 3 | precondition files, testplan files, etc. 4 | 5 | It is here to complement the manual without flooding 6 | it. Maybe once in a while we migrate content over to 7 | the manual. 8 | -------------------------------------------------------------------------------- /examples/dpath/.gitignore: -------------------------------------------------------------------------------- 1 | *.png 2 | *.tmp 3 | *.html 4 | -------------------------------------------------------------------------------- /examples/dpath/CTCS_ratio.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <_ratio.png" 17 | set yrange [0:110] 18 | 19 | plot '-' using 1:2 with linespoints lt 3 lw 1 title "ratio" 20 | 21 | % my @time = reportdata '{ "suite_name" => "CTCS" } :: /report/id'; 22 | % my @ratio = reportdata '{ "suite_name" => "CTCS" } :: //success_ratio'; 23 | % use Data::Dumper; 24 | % foreach (0..scalar @time) { 25 | % if (my $t=$time[$_] and my $r=$ratio[$_]) { 26 | % #my ($x, $y) = $t =~ m/\d{4}-(\d+)-(\d+)/g; 27 | <% $t %> <% $r %> 28 | % } 29 | % } 30 | 31 | EOTEMPLATE 32 | -------------------------------------------------------------------------------- /examples/dpath/KernBench_ratio.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <_ratio.png" 17 | set yrange [0:110] 18 | 19 | plot '-' using 0:2 with steps lt 3 lw 1 title "ratio" 20 | 21 | % my @time = reportdata '{ "suite_name" => "KernBench" } :: /report/created_at_ymd_hms'; 22 | % my @ratio = reportdata '{ "suite_name" => "KernBench" } :: //success_ratio'; 23 | % use Data::Dumper; 24 | % foreach (0..scalar @ratio) { 25 | % if (my $t=$time[$_] and my $r=$ratio[$_]) { 26 | % my ($x, $y) = $t =~ m/\d{4}-(\d+)-(\d+)/g; 27 | "<% $t %>" <% $r %> 28 | % } 29 | % } 30 | 31 | EOTEMPLATE 32 | -------------------------------------------------------------------------------- /examples/dpath/Tapper_ratio.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < "Tapper-Reports-API" } :: //success_ratio'; #stats-proc-interrupts-before/'; 19 | % foreach (@res) { 20 | <% $_ %> 21 | % } 22 | 23 | EOTEMPLATE 24 | -------------------------------------------------------------------------------- /examples/dpath/benchmark_perlformance_data.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < "benchmark-perlformance" } :: /'; 5 | <% Dumper(\@res) %> 6 | EOTEMPLATE 7 | -------------------------------------------------------------------------------- /examples/dpath/between.mas: -------------------------------------------------------------------------------- 1 | #! mason <today(); 6 | % my $dt2 = DateTime->today()->subtract( hours => 12); 7 | % my @test = reportdata "{ suite_name => {'like', 'Topic-%'}, created_at => { -between => [ '$dt2' , '$dt1' ] } } :: /"; 8 | <% Dumper(@test) %> 9 | EOTEMPLATE 10 | -------------------------------------------------------------------------------- /examples/dpath/context_meta_info.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <parse_datetime("today at midnight"); 6 | % my $from = $today->clone->subtract( days => 0.5 ); 7 | 8 | From: <% $from %> .. <% $today %> 9 | --------------------------------- 10 | 11 | % my $SUITE = "CTCS"; 12 | # ----------------------------------------------- 13 | # F=context_meta_info ; netcat tapper 7358 < $F.mas 14 | # ----------------------------------------------- 15 | 16 | Single report meta information: 17 | ------------------------------- 18 | 19 | %# % my @meta = reportdata "{ suite_name => 'CTCS', created_at => { '>', '$from' } } :: /results//tap/../meta"; 20 | %# 21 | %# % foreach (@meta) { 22 | %# <% Dumper($_) %> 23 | %# % } 24 | 25 | Complete group context information for each report: 26 | --------------------------------------------------- 27 | 28 | % my @context = reportdata "{ id => '21439' } :: /groupcontext"; 29 | 30 | % foreach (@context) { 31 | <% Dumper($_) %> 32 | % } 33 | 34 | 35 | EOTEMPLATE 36 | -------------------------------------------------------------------------------- /examples/dpath/debug_api.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < "KernBench" } :: //section//stats-proc-interrupts-before/'; 5 | % my @all = reportdata '{ id => "18942" } :: /';#/section//stats-proc-interrupts-before/'; 6 | % use Data::Dumper; 7 | % foreach (@all) { 8 | <% Dumper($_) %> 9 | % } 10 | 11 | EOTEMPLATE 12 | -------------------------------------------------------------------------------- /examples/dpath/download-reports.pl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env perl 2 | 3 | use common::sense; 4 | use IO::Socket::INET; 5 | 6 | die "Too few arguments. 7 | Usage: $0 suite_name file_name\n" unless @ARGV > 1; 8 | my $file = $ARGV[1]; 9 | 10 | my $search = q(#! mason debug=1 < '); 12 | $search .= $ARGV[0]; 13 | $search .= q('} :: /"; 14 | % use Data::Dumper; 15 | % foreach my $result (@results) { 16 | <% $result->{report}->{id} %> 17 | % } 18 | EOTEMPLATE 19 | ); 20 | 21 | my $sock = IO::Socket::INET->new(PeerAddr => 'tapper', 22 | PeerPort => '7358', 23 | Proto => 'tcp') or die "Can not open socket: $!\n"; 24 | $sock->print($search); 25 | 26 | my @reports; 27 | while (my $line = <$sock>){ 28 | push @reports,int($line); 29 | } 30 | close $sock; 31 | 32 | foreach my $report ( @reports ) { 33 | $sock = IO::Socket::INET->new(PeerAddr => 'localhost', 34 | PeerPort => '7358', 35 | Proto => 'tcp') or die "Can not open socket: $!\n"; 36 | $sock->print("#! download $report $file\n"); 37 | open my $fh, ">", "$file-$report" or die "Can't open ./$file-$report: $!" ; 38 | while (my $line = <$sock>) { 39 | print $fh $line; 40 | } 41 | close $sock; 42 | close $fh; 43 | say STDERR "Wrote ./$file-$report"; 44 | } 45 | -------------------------------------------------------------------------------- /examples/dpath/get_known_good_xen_unstable.mas: -------------------------------------------------------------------------------- 1 | #! mason <now()->subtract( weeks => 1); 5 | % my $dt1 = DateTime->now(); 6 | % my @collected_data; 7 | % @collected_data = reportdata "{suite_name => {'like', 'Topic-xen-unstable'}, created_at => { -between => ['$dt2' , '$dt1' ] } } :: //section/MCP-overview//summary/status[value ~~ 'PASS']/../../../meta/reportgroup-testrun"; 8 | <% $collected_data[0] %> 9 | EOTEMPLATE 10 | -------------------------------------------------------------------------------- /examples/dpath/get_precondition.mas: -------------------------------------------------------------------------------- 1 | #! mason <now()->subtract( weeks => 1); 7 | % my $dt1 = DateTime->now(); 8 | % my @collected_data; 9 | % @collected_data = testrundata "{id => 122345 } :: //precondition"; 10 | <% $collected_data[0] %> 11 | EOTEMPLATE 12 | -------------------------------------------------------------------------------- /examples/dpath/groupstats.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < 23132 } :: /groupstats'; 10 | % foreach my $t (@time) { 11 | <% Dumper($t) %> 12 | % } 13 | 14 | % @time = reportdata '{ id => 23132 } :: /'; 15 | % foreach my $t (@time) { 16 | <% Dumper($t) %> 17 | % } 18 | EOTEMPLATE 19 | -------------------------------------------------------------------------------- /examples/dpath/hello.mas: -------------------------------------------------------------------------------- 1 | #! mason <! 4 | % my @res = reportdata '{ "suite.name" => "power_msr_test" } :: //tap/tests_planned'; 5 | Planned tests: 6 | % foreach (@res) { 7 | <% $_ %> 8 | % } 9 | EOTEMPLATE 10 | -------------------------------------------------------------------------------- /examples/dpath/hello.tt: -------------------------------------------------------------------------------- 1 | #! tt debug=1 < "power_msr_test" } :: //tap/tests_planned') -%] 4 | [% x %] 5 | [% END %] 6 | EOTEMPLATE 7 | -------------------------------------------------------------------------------- /examples/dpath/limit_by_date.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <parse_datetime("today at midnight"); 6 | % my $from = $today->clone->subtract( days => 2 ); 7 | 8 | From: <% $from %> .. <% $today %> 9 | --------------------------------- 10 | 11 | % my $SUITE = "CTCS"; 12 | # ----------------------------------------------- 13 | # F=limit_by_date ; netcat tapper 7358 < $F.mas 14 | # ----------------------------------------------- 15 | 16 | % my @ratio = reportdata "{ suite_name => 'CTCS', created_at => { '>', '$from' } } :: /report/success_ratio"; 17 | 18 | % foreach my $r (@ratio) { 19 | <% $r %> 20 | % } 21 | EOTEMPLATE 22 | -------------------------------------------------------------------------------- /examples/dpath/metadata_recovery.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import re 4 | import simplejson 5 | import socket 6 | import yaml 7 | import urllib2 8 | import sys 9 | mason = '''#! mason debug=1 <new(year => 2010, month=>2, day =>8 ); 12 | % my $dt2 = DateTime->new( year => 2010, month=>3, day =>14 ); 13 | % my $counter = 0; 14 | % my %summary = (); 15 | % my @t = reportdata "{ suite_name => { 'like', [ 'Topic-xen%' ]}, created_at => { -between => ['$dt1' , '$dt2' ] }} :: /"; 16 | % foreach my $run (@t){ 17 | % my $tmp_meta = $run ~~ dpath "//meta//XEN-Metainfo"; 18 | % if ( ! $tmp_meta->[0] ){ 19 | % my $testrun = ($run ~~ dpath "//groupcontext/testrun/")->[0]; 20 | % my $runid = (keys %$testrun)[0]; 21 | % my $machine = ($run ~~ dpath "//hostname")->[0]; 22 | % $summary{ $counter } = [ $runid, $machine ]; 23 | % $counter++; 24 | % } 25 | % } 26 | <% to_json(\%summary) %> 27 | EOTEMPLATE 28 | ''' 29 | 30 | def send_template(template, port): 31 | ''' 32 | send the template to the receiver api 33 | ''' 34 | 35 | s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 36 | received_data = '' 37 | try: 38 | s.connect(("165.204.15.71", port)) 39 | s.send(template) 40 | if port == 7358: 41 | while 1: 42 | data = s.recv(4096) 43 | if not data: 44 | break 45 | received_data += data 46 | return_data = simplejson.loads(received_data) 47 | return return_data 48 | finally: 49 | s.close() 50 | 51 | def fetch_precond(runid): 52 | url = "http://tapper/tapper/testruns/%s/preconditions/yaml/" % runid 53 | response = urllib2.urlopen(urllib2.Request(url)) 54 | precond = response.read() 55 | return precond 56 | 57 | 58 | machines = { 59 | "uruk": { "type":"warthog", 60 | "ram":"12288 MB", 61 | "cpu":"2x Family: 16, Model: 2, Stepping: 3"}, 62 | "incubus": { "type":"warthog", 63 | "ram":"6144 MB", 64 | "cpu":""}, 65 | "kobold": { "type":"Cheatah", 66 | "ram":"5122 MB", 67 | "cpu":"2x Family: 16, Model: 2, Stepping: 2"}, 68 | "satyr": { "type":"PC-Ware", 69 | "ram":"8192 MB", 70 | "cpu":"1x Family: 15, Model: 75, Stepping: 2"}, 71 | "lemure": { "type":"PC-Ware", 72 | "ram":"4096 MB", 73 | "cpu":"1x Family: 15, Model: 107, Stepping: 1"}, 74 | "azael": { "type":"PC-Ware", 75 | "ram":"8192 MB", 76 | "cpu":"1x Family: 15, Model: 75, Stepping: 2"}, 77 | "athene": { "type":"PC-Ware", 78 | "ram":"8192 MB", 79 | "cpu":"1x Family: 15, Model: 75, Stepping: 2"}, 80 | } 81 | 82 | 83 | testdata = send_template(mason, 7358) 84 | for key in testdata.keys(): 85 | runid = testdata[key][0] 86 | precond = yaml.load(fetch_precond(runid)) 87 | arch = precond["host"]["preconditions"][0]["filename"].rsplit("/",3)[-3] 88 | changeset = ":".join(precond["host"]["preconditions"][0]["filename"].rsplit("/",1)[-1].rsplit(".",3)[-3].split("_",2)[:2]) 89 | xenversion = precond["host"]["preconditions"][0]["filename"].rsplit("/",2)[-2] 90 | dom0_kernel = "2.6.18.8-xen %s" % (arch) 91 | base_image = "%s (%s)" % (precond["host"]["root"]["image"].rsplit("/")[-1].rsplit("_", 3)[0].replace( 92 | "_", " ").upper(), arch) 93 | host = testdata[key][1] 94 | 95 | # host meta 96 | tapreport = "1..1\n# Tapper-section: Metainfo\n" 97 | tapreport += "# Tapper-Suite-Name: Host-Overview\n" 98 | tapreport += "# Tapper-Suite-version: manual\n" 99 | tapreport += "# Tapper-machine-name: %s\n" % host 100 | tapreport += "# Tapper-machine-description: %s\n" % ( 101 | machines[host]["type"]) 102 | tapreport += "# Tapper-cpuinfo: %s\n" % ( 103 | machines[host]["cpu"]) 104 | tapreport += "# Tapper-ram:%s \n" % ( 105 | machines[host]["ram"]) 106 | tapreport += "# Tapper-uptime: 0 hrs\n" 107 | tapreport += "# Tapper-reportgroup-testrun: %s \n" % runid 108 | tapreport += "ok 1 host metadata\n" 109 | 110 | # xen meta 111 | tapreport += "1..1\n# Tapper-section: XEN-Metainfo\n" 112 | tapreport += "# Tapper-xen-version: %s \n" % xenversion 113 | tapreport += "# Tapper-xen-changeset: %s\n" % changeset 114 | tapreport += "# Tapper-xen-base-os-description: %s\n" % base_image 115 | tapreport += "# Tapper-xen-dom0-kernel: %s\n" % dom0_kernel 116 | tapreport += "ok 1 xen metadata\n" 117 | # guest meta 118 | for guest in precond["guests"]: 119 | image = guest["root"]["name"].rsplit("/",1)[-1].rsplit(".",1)[0] 120 | tapreport += "1..1\n# Tapper-section: %s\n" % image 121 | count = precond["guests"].index(guest) + 1 122 | desc = guest["testprogram"]["execname"].rsplit("/",1)[-1].split("_")[1] 123 | tapreport += "# Tapper-xen-guest-description: 00%s-%s\n" % (count, 124 | desc) 125 | tapreport += "# Tapper-xen-guest-start: not available\n" 126 | tapreport += "# Tapper-xen-guest-flags: not available\n" 127 | tapreport += "ok 1 - Guest info\n" 128 | 129 | send_template(tapreport, 7357) 130 | 131 | 132 | # vim:set ft=python et ts=4 sw=4 sts=4 sta ai si tw=78: 133 | 134 | -------------------------------------------------------------------------------- /examples/dpath/metareports/ctcs_summary_failed_tests.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < '$SUITE', } :: //CTCS-results/tap/lines/*/description/../"){ 9 | % my $description = $rep ~~ dpath "//description"; 10 | % foreach my $desc (@$description){ 11 | % my $path = "//description[value eq '$desc']/../is_ok/"; 12 | % my $result = $rep ~~ dpath $path; 13 | % $statistic{$desc}++; 14 | % if ( $result->[0] ){ 15 | % $fail{$desc} += 0; 16 | % } else { 17 | % $fail{$desc}++; 18 | % } 19 | % } 20 | % } 21 | %# gnuplot area 22 | set term postscript eps enhanced color 23 | set title "<% $SUITE %>: performed vs. failed subtests\nCreated: `date +%F`" 24 | set style histogram rows 25 | set style fill solid .9 border -1 26 | set term png size 1024, 768 27 | set output "`echo $NFS_DIR`/Misc/CTCS/`date +%F`_<% $SUITE %>_tests_performed_vs_failed.png" 28 | set xtic auto rotate by 90 29 | set ytic auto 30 | %# generate temp data file 31 | set print "<% $SUITE %>_data.tmp" 32 | % my $count = 1; 33 | % foreach my $k (keys (%statistic)){ 34 | print "<% $count %> <% $k %> <% $statistic{$k} %> <% $fail{$k} %>" 35 | % $count += 1; 36 | %} 37 | unset print 38 | %# need some range adjustments to prevent cutoffs on the etches 39 | % my $range = (scalar keys %statistic) + 1; 40 | set xrange [-1:<% $range %>] 41 | 42 | plot '<% $SUITE %>_data.tmp' using 4:xticlabels(3) w boxes lc rgb 'green' title "performed", \ 43 | '' using 5 w boxes lc rgb 'red' title "failed" 44 | 45 | EOTEMPLATE 46 | 47 | # vim:set ft=mason et ts=2 sw=2 sts=2 sta ai si tw=100: 48 | -------------------------------------------------------------------------------- /examples/dpath/metareports/tapper_rotation_overview_1day.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <today(); 12 | % my $dt2 = DateTime->today()->subtract( days => $duration ); 13 | set print "data.tmp" 14 | print "Testsuite failed passed" 15 | % foreach my $suite (@test_suites){ 16 | % my $rscore = "{'suite_name' => '$suite', 'created_at' => { -between => [ '$dt2' , '$dt1' ] } }"; 17 | % my $counter = 0; 18 | % my $fail = 0; 19 | % my @suite_score = reportdata "$rscore :: //success_ratio"; 20 | % foreach my $score (@suite_score){ 21 | % $counter += 1; 22 | % if ($score < 100){ 23 | % $fail += 1; 24 | % } 25 | % } 26 | % 27 | %# create summary in tmp file 28 | % $summary{ $suite } = [ $counter, $fail ]; 29 | print "<% $suite %> <% $fail %> <% $counter %> 30 | % } 31 | unset print 32 | % 33 | %# gnuplot section 34 | set title "TAPPER Overview: performed vs. failed tests\nDuration: last 24hrs" 35 | set terminal postscript eps enhanced 36 | set term png size 1024, 768 37 | set bar 1.000000 38 | set border 3 front linetype -1 linewidth 1.000 39 | set boxwidth 0.75 absolute 40 | set style fill solid 1.00 border -1 41 | set style rectangle back fc lt -3 fillstyle solid 1.00 border -1 42 | set grid nopolar 43 | set grid noxtics nomxtics ytics nomytics noztics nomztics nox2tics nomx2tics noy2tics nomy2tics nocbtics nomcbtics 44 | set grid layerdefault linetype 0 linewidth 1.000, linetype 0 linewidth 1.000 45 | set key outside right top vertical Left reverse enhanced autotitles columnhead nobox 46 | set key invert samplen 4 spacing 1 width 0 height 0 47 | set style histogram rowstacked title offset character 0, 0, 0 48 | set datafile missing '-' 49 | set style data histograms 50 | set xtics border in scale 1,0.5 nomirror offset character 0, 0, 0 51 | set xtics norangelimit 52 | set ytics auto 53 | set rrange [ * : * ] noreverse nowriteback # (currently [0.00000:10.0000] ) 54 | set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 55 | set urange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 56 | set vrange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 57 | set ylabel "% of total" 58 | set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 59 | set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 60 | set yrange [ 0.00000 : 100.000 ] noreverse nowriteback 61 | set cblabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 62 | set output "`echo $NFS_DIR`/Rotation_overview/01_daily/`date +%F`_tapper_overview_last_24hrs.png" 63 | 64 | set print "stringvar.tmp" 65 | print "Results" 66 | % foreach my $test (@test_suites){ 67 | print "<% $test %>: <% $summary{$test}->[1] %> of <% $summary{$test}->[0] %> failed" 68 | % } 69 | unset print 70 | set label 1 system("cat stringvar.tmp") at graph 0.93, graph 0.76 71 | 72 | plot 'data.tmp' using (100.*$2/$3):xtic(1) title column(2), '' using (100.*$3/$3) title column 73 | 74 | EOTEMPLATE 75 | -------------------------------------------------------------------------------- /examples/dpath/metareports/tapper_rotation_overview_1month.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <today(); 12 | % my $dt2 = DateTime->today()->subtract( months => $duration ); 13 | set print "data.tmp" 14 | print "Testsuite failed passed" 15 | % foreach my $suite (@test_suites){ 16 | % my $rscore = "{'suite_name' => '$suite', 'created_at' => { -between => [ '$dt2' , '$dt1' ] } }"; 17 | % my $counter = 0; 18 | % my $fail = 0; 19 | % my @suite_score = reportdata "$rscore :: //success_ratio"; 20 | % foreach my $score (@suite_score){ 21 | % $counter += 1; 22 | % if ($score < 100){ 23 | % $fail += 1; 24 | % } 25 | % } 26 | % 27 | %# create summary in tmp file 28 | % $summary{ $suite } = [ $counter, $fail ]; 29 | print "<% $suite %> <% $fail %> <% $counter %> 30 | % } 31 | unset print 32 | % 33 | %# gnuplot section 34 | set title "TAPPER Overview: performed vs. failed tests\nDuration: <% $dt1->ymd %> - <% $dt2-ymd %>\nCreated: `date +%F`" 35 | set terminal postscript eps enhanced 36 | set term png size 1024, 768 37 | set bar 1.000000 38 | set border 3 front linetype -1 linewidth 1.000 39 | set boxwidth 0.75 absolute 40 | set style fill solid 1.00 border -1 41 | set style rectangle back fc lt -3 fillstyle solid 1.00 border -1 42 | set grid nopolar 43 | set grid noxtics nomxtics ytics nomytics noztics nomztics nox2tics nomx2tics noy2tics nomy2tics nocbtics nomcbtics 44 | set grid layerdefault linetype 0 linewidth 1.000, linetype 0 linewidth 1.000 45 | set key outside right top vertical Left reverse enhanced autotitles columnhead nobox 46 | set key invert samplen 4 spacing 1 width 0 height 0 47 | set style histogram rowstacked title offset character 0, 0, 0 48 | set datafile missing '-' 49 | set style data histograms 50 | set xtics border in scale 1,0.5 nomirror offset character 0, 0, 0 51 | set xtics norangelimit 52 | set ytics auto 53 | set rrange [ * : * ] noreverse nowriteback # (currently [0.00000:10.0000] ) 54 | set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 55 | set urange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 56 | set vrange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 57 | set ylabel "% of total" 58 | set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 59 | set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 60 | set yrange [ 0.00000 : 100.000 ] noreverse nowriteback 61 | set cblabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 62 | set output "`echo $NFS_DIR`/Rotation_overview/03_monthly/`date +%F`_tapper_overview_last_month.png" 63 | 64 | set print "stringvar.tmp" 65 | print "Results" 66 | % foreach my $test (@test_suites){ 67 | print "<% $test %>: <% $summary{$test}->[1] %> of <% $summary{$test}->[0] %> failed" 68 | % } 69 | unset print 70 | set label 1 system("cat stringvar.tmp") at graph 0.93, graph 0.76 71 | 72 | plot 'data.tmp' using (100.*$2/$3):xtic(1) title column(2), '' using (100.*$3/$3) title column 73 | 74 | EOTEMPLATE 75 | -------------------------------------------------------------------------------- /examples/dpath/metareports/tapper_rotation_overview_1week.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <today(); 12 | % my $dt2 = DateTime->today()->subtract( days => $duration ); 13 | set print "data.tmp" 14 | print "Testsuite failed passed" 15 | % foreach my $suite (@test_suites){ 16 | % my $rscore = "{'suite_name' => '$suite', 'created_at' => { -between => [ '$dt2' , '$dt1' ] } }"; 17 | % my $counter = 0; 18 | % my $fail = 0; 19 | % my @suite_score = reportdata "$rscore :: //success_ratio"; 20 | % foreach my $score (@suite_score){ 21 | % $counter += 1; 22 | % if ($score < 100){ 23 | % $fail += 1; 24 | % } 25 | % } 26 | % 27 | %# create summary in tmp file 28 | % $summary{ $suite } = [ $counter, $fail ]; 29 | print "<% $suite %> <% $fail %> <% $counter %> 30 | % } 31 | unset print 32 | % 33 | %# gnuplot section 34 | set title "TAPPER Overview: performed vs. failed tests\nDuration: <% $dt1->ymd %> - <% $dt2-ymd %>\nCreated: `date +%F`" 35 | set terminal postscript eps enhanced 36 | set term png size 1024, 768 37 | set bar 1.000000 38 | set border 3 front linetype -1 linewidth 1.000 39 | set boxwidth 0.75 absolute 40 | set style fill solid 1.00 border -1 41 | set style rectangle back fc lt -3 fillstyle solid 1.00 border -1 42 | set grid nopolar 43 | set grid noxtics nomxtics ytics nomytics noztics nomztics nox2tics nomx2tics noy2tics nomy2tics nocbtics nomcbtics 44 | set grid layerdefault linetype 0 linewidth 1.000, linetype 0 linewidth 1.000 45 | set key outside right top vertical Left reverse enhanced autotitles columnhead nobox 46 | set key invert samplen 4 spacing 1 width 0 height 0 47 | set style histogram rowstacked title offset character 0, 0, 0 48 | set datafile missing '-' 49 | set style data histograms 50 | set xtics border in scale 1,0.5 nomirror offset character 0, 0, 0 51 | set xtics norangelimit 52 | set ytics auto 53 | set rrange [ * : * ] noreverse nowriteback # (currently [0.00000:10.0000] ) 54 | set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 55 | set urange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 56 | set vrange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 57 | set ylabel "% of total" 58 | set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 59 | set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 60 | set yrange [ 0.00000 : 100.000 ] noreverse nowriteback 61 | set cblabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 62 | set output "`echo $NFS_DIR`/Rotation_overview/02_weekly/`date +%F`_tapper_overview_last_week.png" 63 | 64 | set print "stringvar.tmp" 65 | print "Results" 66 | % foreach my $test (@test_suites){ 67 | print "<% $test %>: <% $summary{$test}->[1] %> of <% $summary{$test}->[0] %> failed" 68 | % } 69 | unset print 70 | set label 1 system("cat stringvar.tmp") at graph 0.93, graph 0.76 71 | 72 | plot 'data.tmp' using (100.*$2/$3):xtic(1) title column(2), '' using (100.*$3/$3) title column 73 | 74 | EOTEMPLATE 75 | -------------------------------------------------------------------------------- /examples/dpath/metareports/tapper_rotation_overview_alltime.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < <% $fail %> <% $counter %> 27 | % } 28 | unset print 29 | % 30 | % my $anzahl = (scalar @test_suites); 31 | %# gnuplot section 32 | set title "TAPPER Overview: performed vs. failed tests\nAlltime Overview\nCreated: `date +%F`" 33 | set terminal postscript eps enhanced 34 | set term png size 1024, 768 35 | set bar 1.000000 36 | set border 3 front linetype -1 linewidth 1.000 37 | set boxwidth 0.75 absolute 38 | set style fill solid 1.00 border -1 39 | set style rectangle back fc lt -3 fillstyle solid 1.00 border -1 40 | set grid nopolar 41 | set grid noxtics nomxtics ytics nomytics noztics nomztics nox2tics nomx2tics noy2tics nomy2tics nocbtics nomcbtics 42 | set grid layerdefault linetype 0 linewidth 1.000, linetype 0 linewidth 1.000 43 | set key outside right top vertical Left reverse enhanced autotitles columnhead nobox 44 | set key invert samplen 4 spacing 1 width 0 height 0 45 | set style histogram rowstacked title offset character 0, 0, 0 46 | set datafile missing '-' 47 | set style data histograms 48 | set xtics border in scale 1,0.5 nomirror offset character 0, 0, 0 49 | set xtics norangelimit 50 | set ytics auto 51 | set rrange [ * : * ] noreverse nowriteback # (currently [0.00000:10.0000] ) 52 | set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 53 | set urange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 54 | set vrange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) 55 | set ylabel "% of total" 56 | set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 57 | set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 58 | set yrange [ 0.00000 : 100.000 ] noreverse nowriteback 59 | set cblabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by 90 60 | set output "`echo $NFS_DIR`/Rotation_overview/04_alltime/`date +%F`_tapper_overview_alltime.png" 61 | 62 | set print "stringvar.tmp" 63 | print "Results" 64 | % foreach my $test (@test_suites){ 65 | print "<% $test %>: <% $summary{$test}->[1] %> of <% $summary{$test}->[0] %> failed" 66 | % } 67 | unset print 68 | set label 1 system("cat stringvar.tmp") at graph 0.93, graph 0.76 69 | 70 | plot 'data.tmp' using (100.*$2/$3):xtic(1) title column(2), '' using (100.*$3/$3) title column 71 | 72 | EOTEMPLATE 73 | -------------------------------------------------------------------------------- /examples/dpath/metareports/tapper_tests_per_week.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <weekly; 5 | % my $dt_start = DateTime->new( year => 2009, month => 1, day => 1 ); 6 | % my $dt_end = $dt_start->clone()->add( years => 1 ); 7 | % my @days = $weekly->as_list( start => $dt_start, end => $dt_end ); 8 | % my $week = 1; 9 | % my @result; 10 | % foreach my $date (@days){ 11 | % my $suite_search = "suite_name => { 'like', [ 'Topic-%' ] }"; 12 | % my $next = $weekly->next($date)->clone(); 13 | % my $duration_search = "created_at => { '-between' => [ '$date' , '$next' ] }"; 14 | % my @collected_data = reportdata "{ $suite_search, $duration_search } :: //report/id/"; 15 | % if (scalar @collected_data > 0){ 16 | % push @result, scalar @collected_data; 17 | % } else { 18 | % push @result, 0; 19 | % } 20 | % $week++; 21 | % } 22 | %# gnuplot section 23 | set print 'data.tmp' 24 | print "Week Testruns" 25 | % foreach (1..$week){ 26 | % my $w = $_ - 1; 27 | print "<% $_ %> <% $result[$w] %>" 28 | % } 29 | unset print 30 | set title "TAPPER: Performed tests per Week\nPeriod: <% $dt_start->ymd %> - <% $dt_end->ymd %>\nCreated: `date +%F`" 31 | set terminal postscript eps enhanced 32 | set term png size 1024, 768 33 | set key autotitle columnhead 34 | set boxwidth 0.8 35 | set xrange [ 0 : <% $week + 1 %> ] 36 | set xlabel "Week" 37 | set ylabel "Testruns" 38 | set xtics 5 39 | set ytics 25 40 | 41 | set print "stringvar.tmp" 42 | print "Incidents" 43 | print "Week 05: Tapper goes live" 44 | print "Week 12: misc" 45 | print "Week 18/19: LAB-Net" 46 | print "Week 34: MCP" 47 | print "Week 39: PXE-boot" 48 | unset print 49 | set label 1 system("cat stringvar.tmp") at graph 0.05, graph 0.95 50 | 51 | set output "`echo $NFS_DIR`/Misc/Tests_per_week/`date +%F`_test_overview.png" 52 | plot 'data.tmp' using 1:2 ti col w boxes fs solid 1 ls 2 53 | 54 | EOTEMPLATE 55 | # vim:set ft=mason et ts=2 sw=2 sts=2 sta ai si tw=100: 56 | -------------------------------------------------------------------------------- /examples/dpath/metareports/tapper_topic_ratio.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason DEBUG=1 <today()->subtract( months => $duration ); 7 | % my $dt2 = DateTime->today()->add( days => 1 ); 8 | % # define the searches 9 | % my $suite_search = "suite_name => { 'like', [ '$topics' ] }"; 10 | % my $duration_search = "created_at => { '-between' => [ '$dt1' , '$dt2' ] }"; 11 | % my @collected_data = reportdata "{ $suite_search, $duration_search } :: /report"; 12 | % my %results; 13 | % # collect the data and put into structure 14 | % foreach my $run_temp (@collected_data){ 15 | % my $ratio = $run_temp ~~ dpath "//success_ratio/"; 16 | % my $title = $run_temp ~~ dpath "//suite_name/"; 17 | % $title->[0] =~ s/Topic-//; 18 | % $title->[0] =~ s/-(32|64|default|mix)$//; 19 | % push @{$results{$title->[0]}}, $ratio->[0]; 20 | % } 21 | % # gnuplot area 22 | % while ((my $key, my $value) = each(%results)) { 23 | set print 'data.tmp' 24 | % my $counter = 1; 25 | % my $average = 0; 26 | % my $temp_add = 0; 27 | % foreach my $data (@$value){ 28 | % $temp_add += $data; 29 | % $average = $temp_add/$counter; 30 | print "<% $counter %> <% $data %> <% $average %>" 31 | % $counter++; 32 | % } 33 | unset print 34 | set title "<% $key %>: Success Ratio\nTimeframe: <% $dt1->ymd %> - <% $dt2->ymd %>\nCreated: `date +%F`" 35 | set key bmargin left horizontal Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 36 | set terminal postscript eps enhanced 37 | set term png size 1024, 768 38 | set output "<% $key %>_`date +%F`_success_ratio_<% $key %>.png" 39 | plot 'data.tmp' using 1:2 title "Success Ratio" with imp ls 2, \ 40 | 'data.tmp' using 1:2 smooth bezier title "normalized (bezier)" with lines ls 1, \ 41 | 'data.tmp' using 1:3 title "Average" with lines ls 3 42 | % } 43 | 44 | %#<% Dump(\@collected_data) %> 45 | %#<% Dump($title->[0]) %> 46 | 47 | EOTEMPLATE 48 | # vim:set ft=mason et ts=2 sw=2 sts=2 sta ai si tw=100: 49 | -------------------------------------------------------------------------------- /examples/dpath/metareports/teaser.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <today()->subtract( months => $duration ); 7 | % my $dt2 = DateTime->today()->add( days => 1 ); 8 | % # define the searches 9 | % my $suite_search = "suite_name => { 'like', [ '$topics' ] }"; 10 | % my $duration_search = "created_at => { '-between' => [ '$dt1' , '$dt2' ] }"; 11 | % my @collected_data = reportdata "{ $suite_search, $duration_search } :: /report"; 12 | % my %results; 13 | % # collect the data and put into structure 14 | % foreach my $run_temp (@collected_data){ 15 | % my $ratio = $run_temp ~~ dpath "//success_ratio/"; 16 | % my $title = $run_temp ~~ dpath "//suite_name/"; 17 | % $title->[0] =~ s/Topic-//; 18 | % $title->[0] =~ s/-(32|64|default|mix)$//; 19 | % push @{$results{$title->[0]}}, $ratio->[0]; 20 | % } 21 | % # gnuplot area 22 | % while ((my $key, my $value) = each(%results)) { 23 | set print 'teaser_data.tmp' 24 | % my $counter = 1; 25 | % my $average = 0; 26 | % my $temp_add = 0; 27 | % foreach my $data (@$value){ 28 | % $temp_add += $data; 29 | % $average = $temp_add/$counter; 30 | print "<% $counter %> <% $data %> <% $average %>" 31 | % $counter++; 32 | % } 33 | unset print 34 | # set title "<% $key %>: Success Ratio\nTimeframe: <% $dt1->ymd %> - <% $dt2->ymd %>\nCreated: `date +%F`" 35 | #set title "-" 36 | #set key bmargin left horizontal Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 37 | #set key bmargin left horizontal Right noreverse enhanced box linetype -1 linewidth 1.000 38 | set bmargin 1 39 | unset xtics 40 | unset ytics 41 | #set terminal postscript eps enhanced 42 | set term png size 150, 100 43 | set output "`echo $NFS_DIR`/Topic-ratio/<% $key %>/teaser/`date +%F`_success_ratio_<% $key %>.png" 44 | plot 'teaser_data.tmp' using 1:2 title "" smooth bezier with lines ls 1 45 | % } 46 | 47 | %#<% Dump(\@collected_data) %> 48 | %#<% Dump($title->[0]) %> 49 | 50 | EOTEMPLATE 51 | # vim:set ft=mason et ts=2 sw=2 sts=2 sta ai si tw=100: 52 | -------------------------------------------------------------------------------- /examples/dpath/phoronix.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | ''' 3 | This script connects to the Tapper server and retrieves data from the performed 4 | Phoronix tests in the last 1 week. 5 | ''' 6 | 7 | import socket 8 | import sys 9 | import simplejson as json 10 | import yaml 11 | from statlib import stats 12 | 13 | 14 | def send(perl_template): 15 | ''' 16 | send the template to the receiver api 17 | ''' 18 | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 19 | received_data = '' 20 | try: 21 | sock.connect(("165.204.15.71", 7358)) 22 | sock.send(perl_template) 23 | while 1: 24 | data = sock.recv(1024) 25 | if not data: 26 | break 27 | received_data += data 28 | #print received_data 29 | #sys.exit(0) 30 | return_data = json.loads(received_data) 31 | #print return_data 32 | #sys.exit(0) 33 | return return_data 34 | except socket.gaierror, err: 35 | print "Address-related error connecting to server: %s" % err 36 | sys.exit(1) 37 | except socket.error, err: 38 | print "Connection error: %s" % err 39 | sys.exit(1) 40 | finally: 41 | sock.close() 42 | 43 | def template(): 44 | ''' 45 | This is just a placeholder to part the Python and the Perl code 46 | ''' 47 | mason = '''#! mason debug=1 <today()->subtract(days => 1); 50 | %# my $dt1 = DateTime->new(year => 2010, month=>9, day =>18 ); 51 | % my $dt2 = DateTime->today()->add(days => 1); 52 | % my @runs = reportdata "{ 53 | % suite_name => { 'like', [ 'Phoronix%' ] }, 54 | % created_at => { -between => ['$dt1' , '$dt2' ] }, 55 | % machine_name => { 'like', ['calcium', 'incubus', 'schwertleite'] } 56 | % } :: /"; 57 | % my %sumary; 58 | % foreach my $run (@runs){ 59 | % my $kernel_ver = $run ~~ dpath "//groupcontext//kernel"; 60 | % my $hostname = $run ~~ dpath "//hostname"; 61 | % my $testid = $run ~~ dpath "//id"; 62 | % my @data = $run ~~ dpath "//Phoronix-Results//data/"; 63 | % foreach my $result (@data){ 64 | % my @res= ($hostname->[0], @$result); 65 | % push @{$sumary{$kernel_ver->[0]}{$testid->[0]}}, @res; 66 | % } 67 | % } 68 | <% to_json( unbless \%sumary) %> 69 | EOTEMPLATE 70 | ''' 71 | return mason 72 | 73 | def main(): 74 | ''' 75 | this is the main section 76 | 77 | {kernel: {machine : {[results]} } } 78 | ''' 79 | scales = {} 80 | data = send(template()) 81 | for kernel, results in data.iteritems(): 82 | for testid, run_data in results.iteritems(): 83 | machine = run_data.pop(0) 84 | print "\n%s" % (kernel,) 85 | for i in range(len(run_data)): 86 | for single_bench in run_data[i].itervalues(): 87 | if type(single_bench) == dict: 88 | value = [float(x) for x in 89 | single_bench.get("RawString").split(":")] 90 | val = stats.mean(value) 91 | if len(value) > 1: 92 | std = stats.stdev(value) 93 | else: 94 | std = float(0) 95 | name = single_bench.get("Name") 96 | attr = single_bench.get("Attributes") 97 | scale = single_bench.get("Scale") 98 | print "%s: %s (%s): %.2f %s (std %.2f)" % (machine, 99 | name, attr, val, scale, std) 100 | scales[scale] = scales.get(scale, "") 101 | print scales 102 | if __name__ == '__main__': 103 | main() 104 | 105 | # vim:set sr et tw=80 ts=4 sw=4 st=4 ft=python syn=python fenc=utf-8: 106 | -------------------------------------------------------------------------------- /examples/dpath/plain.mas: -------------------------------------------------------------------------------- 1 | #! mason <! 4 | AFFE 5 | -------------------------------------------------------------------------------- /examples/dpath/proc-interrupts.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < 'KernBench' } :: /results/*/section/stats-proc-interrupts-before/tap/lines/*/_children/*/data/$TOPIC/../../../../../../../../../../../"; 26 | %#% @res = reportdata "{ created_at => { '>', '2009-05-01 00:00' } } :: /results/*/section/stats-proc-interrupts-before/tap/lines/*/_children/*/data/$TOPIC/../../../../../../../../../../../"; 27 | %#% @res = reportdata "{ id => [ -and => { '>', 18870 }, { '<', 18880 } ] } :: /results/*/section/stats-proc-interrupts-before/tap/lines/*/_children/*/data/$TOPIC/../../../../../../../../../../../"; 28 | %#% @res = reportdata "{ suite_name => 'KernBench' } :: /results/*/section/stats-proc-interrupts-before/tap/lines/*/_children/*/data/$TOPIC/../../../../../../../../../../../"; 29 | % @res = reportdata "{ } :: /results/*/section/stats-proc-interrupts-before/tap/lines/*/_children/*/data/$TOPIC/../../../../../../../../../../../"; 30 | %#% @res = ( 18619 ); 31 | %# say STDERR Dumper($res[0]); 32 | % say STDERR "dpath foreach result ... ", ~~localtime; 33 | % foreach (@res) { 34 | % my ($tlb_before) = @{ $_ ~~ dpath "/results/*/section/stats-proc-interrupts-before/tap/lines/*/_children/*/data/$TOPIC/CPU0" }; 35 | % my ($tlb_after) = @{ $_ ~~ dpath "/results/*/section/stats-proc-interrupts-after/tap/lines/*/_children/*/data/$TOPIC/CPU0" }; 36 | % if ($tlb_before and $tlb_after) { 37 | <% $_->{report}{id} %> <% $tlb_after - $tlb_before %> 38 | % } 39 | % } 40 | % say STDERR "done ", ~~localtime; 41 | 42 | EOTEMPLATE 43 | -------------------------------------------------------------------------------- /examples/dpath/python_mason.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | ############# 4 | # Author : Conny Seidel 5 | ############# 6 | 7 | ''' 8 | This is a testscript to gather the needed information in Python via socket and 9 | mason template. 10 | ''' 11 | import sys 12 | import re 13 | import numpy as np 14 | import matplotlib.pyplot as plt 15 | from socket import * 16 | 17 | 18 | mason='''#! mason <new( year => 2009, month => 9,day => 11,hour => 7,minute => 29,second => 58,nanosecond => 0,time_zone => 'GMT',); 23 | %# my $dt2 = DateTime->new( year => 2009, month => 9,day => 11,hour => 7,minute => 30,second => 59,nanosecond => 0,time_zone => 'GMT',); 24 | %# proper timedefinition 25 | % my $dt1 = DateTime->today()->subtract( weeks => 1 ); 26 | % my $dt2 = DateTime->today(); 27 | % my @test = reportdata "{ suite_name => {'like', 'Topic-%'}, created_at => { -between => [ '$dt1' , '$dt2' ] } } :: //xen_changeset/../"; 28 | % foreach my $t (@test) { 29 | % while ( my ($key, $value) = each(%$t) ) { 30 | % print "$value,"; 31 | % } 32 | % print "\\n"; 33 | % } 34 | EOTEMPLATE 35 | ''' 36 | 37 | def send_template(template): 38 | ''' 39 | send the template to the receiver api 40 | ''' 41 | s = socket(AF_INET, SOCK_STREAM) 42 | received_data = '' 43 | try: 44 | s.connect(("165.204.15.71",7358)) 45 | s.send(template) 46 | while 1: 47 | data = s.recv(4096) 48 | if not data: break 49 | received_data += data 50 | finally: 51 | s.close() 52 | return received_data.split('\n') 53 | 54 | 55 | 56 | def parse(data): 57 | ''' 58 | sort the collected data 59 | ''' 60 | summary = { 32 : [], 64 : [] } 61 | for lines in data: 62 | if not data == '': 63 | split = re.sub(r',$','', lines).split(',') 64 | if re.search('.*xen.*i686.*', lines): 65 | summary[32].append(split) 66 | elif re.search('.*xen.*x86_64.*', lines): 67 | summary[64].append(split) 68 | return summary 69 | 70 | def format_results(format): 71 | res = {} 72 | for key in format.keys(): 73 | tmp_dict = {} 74 | for value in format[key]: 75 | if not tmp_dict.has_key(value[1]): 76 | tmp_dict[value[1]] = 1 77 | else: 78 | tmp_dict[value[1]] += 1 79 | res[key] = { 'versions': sorted(tmp_dict.keys()), 80 | 'values': [tmp_dict[i] for i in sorted(tmp_dict.keys())] } 81 | return res[32], res[64] 82 | 83 | test_data = send_template(mason) 84 | parsed_data = parse(test_data) 85 | i386, x86_64 = format_results(parsed_data) 86 | 87 | N = len(i386['values']) 88 | 89 | bit32 = i386['values'] 90 | 91 | ind = np.arange(N) # the x locations for the groups 92 | width = 0.35 # the width of the bars 93 | 94 | fig = plt.figure() 95 | ax = fig.add_subplot(111) 96 | rects1 = ax.bar(ind, bit32, width, color='b') 97 | 98 | bit64 = x86_64['values'] 99 | rects2 = ax.bar(ind+width, bit64, width, color='g') 100 | 101 | # add some 102 | ax.set_ylabel('Tests') 103 | ax.set_title('Overview:\nXEN testing effort') 104 | ax.set_xticks(ind+width) 105 | ax.set_xticklabels( i386['versions'] ) 106 | ax.set_yticks(np.arange(0,60,10)) 107 | 108 | ax.legend( (rects1[0], rects2[0]), ('32 bit', '64 bit') ) 109 | 110 | def autolabel(rects): 111 | # attach some text labels 112 | for rect in rects: 113 | height = rect.get_height() 114 | ax.text(rect.get_x()+rect.get_width()/2., 1.05*height, '%d'%int(height), 115 | ha='center', va='bottom') 116 | 117 | autolabel(rects1) 118 | autolabel(rects2) 119 | 120 | plt.show() 121 | 122 | 123 | 124 | 125 | # vim:set ft=python et ts=4 sw=4 sts=4 sta ai si tw=78: 126 | -------------------------------------------------------------------------------- /examples/dpath/report_dumper.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <today(); 4 | % my $dt2 = DateTime->today()->subtract( weeks => 1); 5 | % my @t = reportdata "{ suite_name => 'Daily-Report', created_at => { -between => ['$dt2' , '$dt1' ] }} :: /"; 6 | % foreach my $run (@t){ 7 | <% Dump(\$run) %> 8 | % } 9 | EOTEMPLATE 10 | -------------------------------------------------------------------------------- /examples/dpath/slbench_evaluate_language_series.pl: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env perl 2 | 3 | use 5.010; 4 | use strict; 5 | use warnings; 6 | 7 | use Data::Dumper; 8 | use Data::DPath "dpath"; 9 | 10 | my @res; 11 | print STDERR "Read data...\n"; 12 | eval qx!cat data.pl!; 13 | print STDERR "Evaluate...\n"; 14 | 15 | sub match { 16 | my ($report, $dpath) = @_; 17 | 18 | # say " $dpath"; 19 | my $result = $report ~~ dpath $dpath; 20 | foreach (@$result) { 21 | $_->{_report_id} = $report->{report}{id} if ref $_ eq "HASH" 22 | } 23 | return $result; 24 | } 25 | 26 | sub res_match { 27 | my ($res, $benchmark_names, $language, $host, $version, $series) = @_; 28 | 29 | my $lazy_result; 30 | foreach my $benchmark (@$benchmark_names) { 31 | foreach my $report (@$res) { 32 | my $_version = $version; $_version =~ s/\./\\./g; 33 | my $_host = $host; $_host =~ s/\./\\./g; 34 | my $dpath = qq'//data//language_series[ value eq "$series"]/../benchmark[ value eq "$benchmark"]/../language_binary[value =~ m!$_version/bin/$language!]/../hostname[value eq "$_host"]/..'; 35 | my $benchvalues = match($report, $dpath); 36 | push @{$lazy_result->{$host}{$language}{$series}{$version}{$benchmark}}, $benchvalues; 37 | } 38 | } 39 | return $lazy_result; 40 | } 41 | 42 | my %benchmarks = map { $_ => 1 } @{ \@res ~~ dpath "//data//mean/../benchmark" }; 43 | my @benchmark_names = keys %benchmarks; 44 | #@benchmark_names = qw(call_method_unknown call_simple); 45 | 46 | my $super_result; 47 | my $lazy_result; 48 | 49 | my $language = "python"; 50 | foreach my $host (qw(elemente)) { 51 | foreach my $version (qw(2.4 2.5 2.6 2.7)) { 52 | foreach my $series (qw(o0 o1 o2 o3)) { 53 | $lazy_result = res_match(\@res, \@benchmark_names, $language, $host, $version, $series); 54 | } 55 | } 56 | } 57 | 58 | # reduce 59 | $language = "python"; 60 | foreach my $host (qw(elemente)) { 61 | foreach my $version (qw(2.4 2.5 2.6 2.7)) { # that's the repetition cause... !? 62 | foreach my $series (qw(o0 o1 o2 o3) ) { 63 | foreach my $benchmark (@benchmark_names) { 64 | foreach (@{$lazy_result->{$host}{$language}{$series}{$version}{$benchmark}}) { 65 | push @{$super_result->{$host}{$language}{$series}{$version}{$benchmark}}, @$_ if @$_; 66 | } 67 | } 68 | } 69 | } 70 | } 71 | 72 | print Dumper($lazy_result); 73 | print Dumper($super_result); 74 | -------------------------------------------------------------------------------- /examples/dpath/suite-percentage.tt: -------------------------------------------------------------------------------- 1 | #! tt debug=1 puresqlabstract=1 < "example-suite-name" } :: //failurecount') %] 9 | [%- FOREACH percent IN failpercentages %] 10 | [% percent %] 11 | [%- END %] 12 | ENDOFTEMPLATE 13 | -------------------------------------------------------------------------------- /examples/dpath/table1.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <today(); 5 | % my $dt2 = DateTime->today()->subtract( weeks => 1); 6 | % my @collected_data = reportdata "{suite_name => {'like', 'Topic-xen%'}, created_at => { -between => ['$dt2' , '$dt1' ] } } :: /"; 7 | % my %results; 8 | % foreach my $run_temp (@collected_data){ 9 | % my $xen_cs = $run_temp ~~ dpath "//xen_changeset"; 10 | % my $xen_vers = $run_temp ~~ dpath "//xen_version"; 11 | % my $ratio = $run_temp ~~ dpath "//success_ratio"; 12 | % my $platform = $run_temp ~~ dpath "//hardwaredb/key_word"; 13 | % my $cpu = $run_temp ~~ dpath "/report//hardwaredb/cpus/*/revision"; 14 | % my $report_id = $run_temp ~~ dpath "//report/id"; 15 | % if ( $xen_cs->[0] ){ 16 | % $xen_cs->[0] =~ s/:.*//; 17 | % $xen_vers->[0] =~ s/-[^u].*//; 18 | % my $platf_str = $platform->[0].' ('.$cpu->[0].')'; 19 | % if ($ratio->[0] == 100){ 20 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{'green'}++; 21 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{'red'} += 0; 22 | % } else { 23 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{'red'}++; 24 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{'green'} += 0; 25 | % } 26 | % push @{$results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{list}}, $report_id->[0] 27 | % } 28 | % } 29 | 30 | %# You choose the output 31 | %#<% Dump(\%results) %> 32 | <% to_json(unbless \%results) %> 33 | 34 | EOTEMPLATE 35 | 36 | 37 | %# output should always look like this 38 | %# table = { 'xen-3.3' => { '1234' => { 'sahara' => { 'red' => 10, 39 | %# 'green' => 4, 40 | %# 'list' => [1,2,3,4,5,......] 41 | %# }, 42 | %# 'cheetah' => {'red' => 0, 43 | %# 'green' => 8, 44 | %# 'list' => [6,7,8,9,10,.....] 45 | %# } 46 | %# }, 47 | %# '1235' => { 'sahara' => { 'red' => 4, 48 | %# 'green' => 9, 49 | %# 'list' => [11,12,13,14,....] 50 | %# }, 51 | %# 'cheetah' => {'red' => 15, 52 | %# 'green' => 10, 53 | %# 'list' => [15,16,17,18,....] 54 | %# } 55 | %# } 56 | %# }, 57 | %# 'xen-3.4' => { '2345' => { 'sahara' => { 'red' => 12, 58 | %# 'green' => 5, 59 | %# 'list' => [20,21,22,23,....] 60 | %# }, 61 | %# 'cheetah' => {'red' => 0, 62 | %# 'green' => 8, 63 | %# 'list' => [25,26,27,28.....] 64 | %# } 65 | %# }, 66 | %# '2346' => { 'sahara' => { 'red' => 4, 67 | %# 'green' => 9, 68 | %# 'list' => [30,31,32,33,....] 69 | %# }, 70 | %# 'cheetah' => {'red' => 15, 71 | %# 'green' => 10, 72 | %# 'list' => [35,36,37,38,....] 73 | %# } 74 | %# } 75 | %# } 76 | %# } 77 | 78 | # vim:set ft=mason et ts=2 sw=2 sts=2 sta ai si tw=100: 79 | -------------------------------------------------------------------------------- /examples/dpath/table2.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 <today()->add( days => 1); 5 | % my $dt2 = DateTime->today()->subtract( days => 7); 6 | 7 | % my @collected_data = reportdata "{suite_name => {'like', [ 'CTCS', 'KernBench', 'LTP', 'LMBench' ] }," . 8 | % " created_at => { -between => ['$dt2' , '$dt1' ] } } :: /"; 9 | % my %results; 10 | % foreach my $run_temp (@collected_data){ 11 | % my $xen_cs = $run_temp ~~ dpath "//xen_changeset"; 12 | % my $xen_vers = $run_temp ~~ dpath "//xen_version"; 13 | % my $ratio = $run_temp ~~ dpath "//success_ratio"; 14 | % my $report_id = $run_temp ~~ dpath "//report/id"; 15 | % my $testrun = $run_temp ~~ dpath "/report/suite_name"; 16 | % if ($xen_cs->[0] ne ""){ 17 | % $xen_cs->[0] =~ s/:.*//; 18 | % $xen_vers->[0] =~ s/-[^u].*//; 19 | % if ($ratio->[0] == 100){ 20 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$testrun->[0]}{'green'}++; 21 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$testrun->[0]}{'red'} += 0; 22 | % } else { 23 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$testrun->[0]}{'red'}++; 24 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$testrun->[0]}{'green'} += 0; 25 | % } 26 | % push @{$results{$xen_vers->[0]}{$xen_cs->[0]}{$testrun->[0]}{list}}, $report_id->[0] 27 | % } 28 | % } 29 | 30 | %# You choose the output 31 | <% Dump(\%results) %> 32 | %#<% to_json(unbless \%results) %> 33 | 34 | EOTEMPLATE 35 | 36 | 37 | %# output should always look like this 38 | %# table = { 'xen-3.3' => { '1234' => { 'LTP' => { 'red' => 10, 39 | %# 'green' => 4, 40 | %# 'list' => [1,2,3,4,5,......] 41 | %# }, 42 | %# 'CTCS' => {'red' => 0, 43 | %# 'green' => 8, 44 | %# 'list' => [6,7,8,9,10,.....] 45 | %# } 46 | %# }, 47 | %# '1235' => { 'LTP' => { 'red' => 4, 48 | %# 'green' => 9, 49 | %# 'list' => [11,12,13,14,....] 50 | %# }, 51 | %# 'CTCS' => {'red' => 15, 52 | %# 'green' => 10, 53 | %# 'list' => [15,16,17,18,....] 54 | %# } 55 | %# } 56 | %# }, 57 | %# 'xen-3.4' => { '2345' => { 'LTP' => { 'red' => 12, 58 | %# 'green' => 5, 59 | %# 'list' => [20,21,22,23,....] 60 | %# }, 61 | %# 'CTCS' => {'red' => 0, 62 | %# 'green' => 8, 63 | %# 'list' => [25,26,27,28.....] 64 | %# } 65 | %# }, 66 | %# '2346' => { 'LTP' => { 'red' => 4, 67 | %# 'green' => 9, 68 | %# 'list' => [30,31,32,33,....] 69 | %# }, 70 | %# 'CTCS' => {'red' => 15, 71 | %# 'green' => 10, 72 | %# 'list' => [35,36,37,38,....] 73 | %# } 74 | %# } 75 | %# } 76 | %# } 77 | -------------------------------------------------------------------------------- /examples/dpath/table_cs_vs_platform.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | ''' 3 | This script connects to the Tapper server and retrieves data from the performed 4 | XEN tests in the last 2 weeks. It will generate a HTML table for each 5 | XEN-Version with XEN-changesets as rows. 6 | ''' 7 | 8 | import socket 9 | import sys 10 | import simplejson 11 | import HTML 12 | 13 | def send(perl_template): 14 | ''' 15 | send the template to the receiver api 16 | ''' 17 | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 18 | received_data = '' 19 | try: 20 | sock.connect(("165.204.15.71", 7358)) 21 | sock.send(perl_template) 22 | while 1: 23 | data = sock.recv(1024) 24 | if not data: 25 | break 26 | received_data += data 27 | return_data = simplejson.loads(received_data) 28 | return return_data 29 | except socket.gaierror, err: 30 | print "Address-related error connecting to server: %s" % err 31 | sys.exit(1) 32 | except socket.error, err: 33 | print "Connection error: %s" % err 34 | sys.exit(1) 35 | finally: 36 | sock.close() 37 | 38 | def template(): 39 | ''' 40 | This is just a placeholder to part the Python and the Perl code 41 | ''' 42 | mason = '''#! mason <today()->subtract( months => 1); 45 | % my $dt2 = DateTime->today(); 46 | % my @collected_data = reportdata "{suite_name => {'like', 'Topic-xen%'}, created_at => { -between => ['$dt1' , '$dt2' ] } } :: /"; 47 | % my %results; 48 | % my @platforms = qw(); 49 | % foreach my $run_temp (@collected_data){ 50 | % my $xen_cs = $run_temp ~~ dpath "//xen_changeset"; 51 | % my $xen_vers = $run_temp ~~ dpath "//xen_version"; 52 | % my $ratio = $run_temp ~~ dpath "//success_ratio"; 53 | % my $platform = $run_temp ~~ dpath "//hardwaredb/key_word"; 54 | % my $cpu = $run_temp ~~ dpath "/report//hardwaredb/cpus/*/revision"; 55 | % my $report_id = $run_temp ~~ dpath "//report/id"; 56 | % my %platforms = map { $_ => 1 } @platforms; 57 | % 58 | % if ( $xen_cs->[0] ){ 59 | % $xen_cs->[0] =~ s/:.*//; 60 | % $xen_vers->[0] =~ s/-[^u].*//; 61 | % my $platf=$platform->[0] || 'unknown'; 62 | % my $cpuf=$cpu->[0] || 'unknown'; 63 | % my $platf_str = $platf.' ('.$cpuf.')'; 64 | % push @platforms, $platf_str if not $platforms{$platf_str}; 65 | % if ($ratio->[0] == 100){ 66 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{'green'}++; 67 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{'red'} += 0; 68 | % } else { 69 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{'red'}++; 70 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{'green'} += 0; 71 | % } 72 | % push @{$results{$xen_vers->[0]}{$xen_cs->[0]}{$platf_str}{list}}, $report_id->[0] 73 | % } 74 | % } 75 | % foreach my $key (keys %results){ 76 | % $results{$key}{"platforms"} = \@platforms; 77 | % } 78 | <% to_json(unbless \%results) %> 79 | EOTEMPLATE 80 | ''' 81 | return mason 82 | 83 | def def_colors(): 84 | ''' 85 | setup colors / ratio 86 | ''' 87 | colors = {0:'red', 50:'orange', 75:'yellow', 95:'greenyellow', 88 | 100:'green'} 89 | return colors 90 | 91 | def legend(): 92 | ''' 93 | create a legend from colors() 94 | ''' 95 | list_col = [] 96 | leg = HTML.TableCell('Legend', bgcolor='lightgray', 97 | width='10%', align='center') 98 | col = def_colors() 99 | for ratio in sorted(col): 100 | if ratio == 0: 101 | cell_txt = str(ratio) + ' %' 102 | else: 103 | cell_txt = '<= %s' % (str(ratio) + '%') 104 | list_col.append(HTML.TableCell(cell_txt, bgcolor=col[ratio], 105 | width='10%', align='center')) 106 | my_legend = HTML.Table(rows=[[leg] + list_col]) 107 | return str(my_legend) 108 | 109 | def link(tmp_list): 110 | ''' 111 | generate links to Tapper 112 | ''' 113 | if len(tmp_list) > 1: 114 | link_html = HTML.link('results+', 115 | 'https://tapper/tapper/reports/idlist/' + 116 | ','.join(str(l) for l in tmp_list)) 117 | elif len(tmp_list) == 1: 118 | link_html = HTML.link('results', 119 | 'https://tapper/tapper/reports/id/' + 120 | ','.join(str(l) for l in tmp_list)) 121 | return link_html 122 | 123 | def main(): 124 | ''' 125 | this is the main section 126 | ''' 127 | data = send(template()) 128 | path = "" 129 | my_file_all = open(path + 'table_XEN_changeset_vs_platform.html', 'w') 130 | for xen_ver in sorted(data): 131 | my_file = open('table_' + xen_ver + '_changeset_vs_platform.html', 'w') 132 | head = HTML.TableRow(cells=([xen_ver] + [i.upper() for i in 133 | sorted(data[xen_ver]['platforms'])]), bgcolor='white', header=True) 134 | my_table = HTML.Table(rows=[head]) 135 | for xen_cs, values in sorted(data[xen_ver].iteritems()): 136 | if xen_cs == 'platforms': 137 | break 138 | list_ratio = [] 139 | for platform in sorted(data[xen_ver]['platforms']): 140 | if values.has_key(platform): 141 | plat = values[platform] 142 | res_link = link(plat['list']) 143 | cols = def_colors() 144 | for rcol in sorted(cols): 145 | if plat['green'] * 100 / (plat['green'] + 146 | plat['red']) >= rcol: 147 | color = cols[rcol] 148 | else: 149 | res_link = 'no tests' 150 | color = 'gray' 151 | list_ratio.append(HTML.TableCell(res_link, bgcolor=color, 152 | width='7%', align='center')) 153 | my_table.rows.append([HTML.TableCell(xen_cs, bgcolor='lightgray', 154 | width='7%', align='center')] + list_ratio) 155 | my_file.write(str(my_table)) 156 | my_file.write(legend()) 157 | my_file.write('

') 158 | my_file_all.write(str(my_table)) 159 | my_file_all.write(legend()) 160 | my_file_all.write('

') 161 | 162 | if __name__ == '__main__': 163 | main() 164 | 165 | # vim:set sr et tw=80 ts=4 sw=4 st=4 ft=python syn=python fenc=utf-8: 166 | 167 | -------------------------------------------------------------------------------- /examples/dpath/table_cs_vs_workload.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | ''' 3 | This script connects to the Tapper server and retrieves data from the performed 4 | XEN tests in the last 2 weeks. It will generate a HTML table for each 5 | XEN-Version with XEN-changesets as rows. 6 | ''' 7 | 8 | import socket 9 | import sys 10 | import simplejson 11 | import HTML 12 | 13 | def send(perl_template): 14 | ''' 15 | send the template to the receiver api 16 | ''' 17 | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 18 | received_data = '' 19 | try: 20 | sock.connect(("165.204.15.71", 7358)) 21 | sock.send(perl_template) 22 | while 1: 23 | data = sock.recv(1024) 24 | if not data: 25 | break 26 | received_data += data 27 | return_data = simplejson.loads(received_data) 28 | return return_data 29 | except socket.gaierror, err: 30 | print "Address-related error connecting to server: %s" % err 31 | sys.exit(1) 32 | except socket.error, err: 33 | print "Connection error: %s" % err 34 | sys.exit(1) 35 | finally: 36 | sock.close() 37 | 38 | def template(): 39 | ''' 40 | This is just a placeholder to part the Python and the Perl code 41 | ''' 42 | mason = '''#! mason <today()->subtract( weeks => 1); 45 | % my $dt2 = DateTime->today()->add( days => 1 ); 46 | % my @collected_data = reportdata "{suite_name => [ -and => { -not_like => 'Topic%' }, " . 47 | % " { -not_like => '%Over%' }, " . 48 | % " { -not_like => 'unknown' }, " . 49 | % " { -not_like => '%Build' } ], " . 50 | % "created_at => { -between => ['$dt1' , '$dt2' ] } } :: /"; 51 | % my %results; 52 | % my @tests = qw(); 53 | % foreach my $run_temp (@collected_data){ 54 | % my $xen_cs = $run_temp ~~ dpath "//xen_changeset"; 55 | % my $xen_vers = $run_temp ~~ dpath "//xen_version"; 56 | % my $ratio = $run_temp ~~ dpath "//report/success_ratio"; 57 | % my $report_id = $run_temp ~~ dpath "/report/id"; 58 | % my $test = $run_temp ~~ dpath "/report/suite_name"; 59 | % my %tests = map { $_ => 1 } @tests; 60 | % if ( $xen_cs->[0] ){ 61 | % $xen_cs->[0] =~ s/:.*//; 62 | % $xen_vers->[0] =~ s/-[^u].*//; 63 | % push @tests, $test->[0] if not $tests{$test->[0]}; 64 | % if ($ratio->[0] == 100){ 65 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$test->[0]}{'green'}++; 66 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$test->[0]}{'red'} += 0; 67 | % } else { 68 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$test->[0]}{'red'}++; 69 | % $results{$xen_vers->[0]}{$xen_cs->[0]}{$test->[0]}{'green'} += 0; 70 | % } 71 | % push @{$results{$xen_vers->[0]}{$xen_cs->[0]}{$test->[0]}{list}}, $report_id->[0] 72 | % } 73 | % foreach my $key (keys %results){ 74 | % $results{$key}{"tests"} = \@tests; 75 | % } 76 | % } 77 | <% to_json( \%results) %> 78 | EOTEMPLATE 79 | ''' 80 | return mason 81 | 82 | def def_colors(): 83 | ''' 84 | setup colors / ratio 85 | ''' 86 | colors = {0:'red', 25:'orange', 50:'yellow', 75:'greenyellow', 87 | 100:'green'} 88 | return colors 89 | 90 | def legend(): 91 | ''' 92 | create a legend from colors() 93 | ''' 94 | list_col = [] 95 | leg = HTML.TableCell('Legend', bgcolor='lightgray', 96 | width='10%', align='center') 97 | col = def_colors() 98 | for ratio in sorted(col): 99 | if ratio == 0: 100 | cell_txt = str(ratio) + ' %' 101 | else: 102 | cell_txt = '<= %s' % (str(ratio) + '%') 103 | list_col.append(HTML.TableCell(cell_txt, bgcolor=col[ratio], 104 | width='10%', align='center')) 105 | my_legend = HTML.Table(rows=[[leg] + list_col]) 106 | return str(my_legend) 107 | 108 | def link(tmp_list): 109 | ''' 110 | generate links to Tapper 111 | ''' 112 | if len(tmp_list) > 1: 113 | link_html = HTML.link('results+', 114 | 'https://tapper/tapper/reports/idlist/' + 115 | ','.join(str(l) for l in tmp_list)) 116 | elif len(tmp_list) == 1: 117 | link_html = HTML.link('results', 118 | 'https://tapper/tapper/reports/id/' + 119 | ','.join(str(l) for l in tmp_list)) 120 | return link_html 121 | 122 | 123 | def main(): 124 | ''' 125 | this is the main section 126 | ''' 127 | data = send(template()) 128 | path = "" 129 | my_file_all = open(path + 'table_XEN_changeset_vs_workload.html', 'w') 130 | for xen_ver in sorted(data): 131 | my_file = open('table_' + xen_ver + '_changeset_vs_workload.html', 'w') 132 | head = HTML.TableRow(cells=([xen_ver] + [i.upper() for i in 133 | sorted(data[xen_ver]['tests'])]), bgcolor='white', header=True) 134 | my_table = HTML.Table(rows=[head]) 135 | for xen_cs, values in sorted(data[xen_ver].iteritems()): 136 | if xen_cs == 'tests': 137 | break 138 | list_ratio = [] 139 | for test in sorted(data[xen_ver]['tests']): 140 | if values.has_key(test): 141 | vtest = values[test] 142 | res_link = link(vtest['list']) 143 | cols = def_colors() 144 | for rcol in sorted(cols): 145 | if vtest['green'] * 100 / ( 146 | vtest['green'] + vtest['red']) >= rcol: 147 | color = cols[rcol] 148 | else: 149 | res_link = 'no tests' 150 | color = 'gray' 151 | list_ratio.append(HTML.TableCell(res_link, bgcolor=color, 152 | width='7%', align='center')) 153 | my_table.rows.append([HTML.TableCell(xen_cs, bgcolor='lightgray', 154 | width='7%', align='center')] + list_ratio) 155 | 156 | my_file.write(str(my_table)) 157 | my_file.write(legend()) 158 | my_file.write('

') 159 | my_file_all.write(str(my_table)) 160 | my_file_all.write(legend()) 161 | my_file_all.write('

') 162 | 163 | if __name__ == '__main__': 164 | main() 165 | 166 | 167 | # vim:set sr et tw=80 ts=4 sw=4 st=4 ft=python syn=python fenc=utf-8: 168 | -------------------------------------------------------------------------------- /examples/dpath/tapper-cmd.mas: -------------------------------------------------------------------------------- 1 | #! mason < "Tapper-Cmd" } :: //section/*//tap/tests_planned'; 5 | Planned tests: 6 | % foreach (@res) { 7 | <% $_ %> 8 | % } 9 | EOTEMPLATE 10 | -------------------------------------------------------------------------------- /examples/dpath/tapper-reports-receiver.mas: -------------------------------------------------------------------------------- 1 | #! mason < "Tapper-Reports-Receiver" } :: //section/*[0]/tap/tests_planned'; 5 | Planned tests: 6 | % foreach (@res) { 7 | <% $_ %> 8 | % } 9 | EOTEMPLATE 10 | -------------------------------------------------------------------------------- /examples/dpath/tapper-schema.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < "Tapper-Schema" } :: //section/"t/tapper_schema_testrundb"//tap/tests_planned'; 5 | Planned tests: 6 | % use Data::Dumper; 7 | % foreach (@res) { 8 | <% Dumper( $_ ) %> 9 | % } 10 | EOTEMPLATE 11 | -------------------------------------------------------------------------------- /examples/dpath/tapper.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < "Tapper" } :: /'; 5 | Planned tests: 6 | % use Data::Dumper; 7 | % foreach (@res) { 8 | <% Dumper( $_ ) %> 9 | % } 10 | EOTEMPLATE 11 | -------------------------------------------------------------------------------- /examples/dpath/warmup.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < 22265 } :: /'; 11 | 12 | % foreach my $t (@time) { 13 | <% Dumper($t) %> 14 | % } 15 | EOTEMPLATE 16 | -------------------------------------------------------------------------------- /examples/dpath/warmup1.gnuplot: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < 14 | 15 | %# <% Dumper($t) %> für Datenstrukturen-Ansicht 16 | %# Eigentliches Ziel: <% $t->{created_at_ymd_hms} %> 17 | 18 | %# Teilziel: verstehe große Datenstruktur unter "/" 19 | %# /report ... Report-Basisdaten 20 | %# /results ... Testresultate 21 | %# Array, jede Zeile ist eine Section: KVM-Metainfo, guest1_rh, guest2_sles, host 22 | 23 | %# Beispielpfad: 24 | %# /results/*/section/"t/00-tapper-meta"/... 25 | %# in "results", irgendeine Zeile, darin key "section", darin key "t/00-tapper-meta" 26 | 27 | %# /results[0]/section/"t/00-tapper-meta"/... 28 | %# in "results", erste Zeile, darin key "section", darin key "t/00-tapper-meta" 29 | 30 | %# /results//section/"t/00-tapper-meta"/... 31 | %# in "results", irgendwelche Zwischenschritte, dann key "section", darin key "t/00-tapper-meta" 32 | 33 | %# "//" ist Killerfeature (leider langsam) 34 | 35 | %# //tap 36 | %# /results//tap/tests_run 37 | 38 | %# Wichtige Pfade: 39 | %# //tap/lines ... darin wieder Array aller TAP-Zeilen 40 | %# //tap/lines/*/is_test[value == 1] ... TAP-Zeilen, die echte Test-Zeilen sind 41 | 42 | %# Typen von TAP-Zeilen: is_version, is_plan, is_test, is_diag, is_yaml 43 | %# //tap/lines/*/raw ... originale TAP-Zeile 44 | 45 | 46 | %# YAML-Blöcke erscheinen wie eine Zeile, also nur ein Element in _children 47 | 48 | %# _children: 49 | %# enthält diag/yaml-Folgezeilen 50 | %# 51 | %# Bsp-TAP: embedded YAML 52 | %# 53 | %# ok 1 - hot stuff 54 | %# --- 55 | %# benchmark_results: 56 | %# graphics: 12.1231 57 | %# math: 32.12 58 | %# ... 59 | %# 60 | %# //tap//description[value eq "- hot stuff"]/../_children/*/data/benchmark_results/graphics 61 | %# //tap//description[value eq "- hot stuff"]/../_children/*/data/benchmark_results/math 62 | %# 63 | %# Alternativ, weiter oben liefern lassen 64 | %# my @result = reportdata '{ ... } :: //tap//description[value eq "- hot stuff"]/..' 65 | %# 66 | %# <% $result[0]->{_children}[0]{data}{benchmark_results}{graphics} %> 67 | %# <% $result[0]->{_children}[0]{data}{benchmark_results}{math} %> 68 | %# 69 | %# 70 | %# 71 | %# Alternative 2: auch *im* Template rohes Data::DPath nutzen: 72 | %# 73 | %# % use Data::DPath 'dpath'; 74 | %# 75 | %# <% $result[0] ~~ '/_children/*/data/benchmark_results/math' %> 76 | %# <% $result[0] ~~ '/_children/*/data/benchmark_results/graphics' %> 77 | %# 78 | %# 79 | %# 80 | %# 81 | %# 82 | %# 83 | %# 84 | 85 | 86 | %# auf YAML im TAP zugreifen: 87 | %# //tap//description[value eq "- hot stuff"]/../_children/*/data/mainboard/socket_type 88 | 89 | %# interessante Reports und Pfade finden: 90 | %# suite_name => "CTCS" 91 | %# section => "rhel5.4_rc1" ... Hinweis auf interessante image 92 | %# 93 | 94 | 95 | % my @time = reportdata '{ id => "13985" } :: /'; 96 | % foreach my $t (@time) { 97 | <% Dumper($t) %> 98 | <% $t->{created_at_ymd_hms} %> 99 | % } 100 | EOTEMPLATE 101 | -------------------------------------------------------------------------------- /examples/dpath/warmup2_json.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < "18257" } :: /'; 8 | 9 | JSON: 10 | <% to_json( unbless \@res ) %> 11 | YAML: 12 | <% Dump(\@res) %> 13 | Dumper: 14 | <% Dumper(\@res) %> 15 | 16 | %# reportgroup_testrun_id not supported yet: 17 | %#% @res = reportdata '{ "reportgroup_testrun_id" => "8809" } :: /report'; 18 | %#JSON: 19 | %#<% to_json(\@res) %> 20 | %# 21 | %#Dumper: 22 | %#<% Dumper(\@res) %> 23 | 24 | EOTEMPLATE 25 | -------------------------------------------------------------------------------- /examples/dpath/warmup3_slbench.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < 3 | # -*- mode: cperl -*- 4 | # F=warmup3_slbench ; netcat tapper 7358 < $F.mas 5 | my @res = reportdata '{ suite_name => "SLBench-Benchmark-Runner" } :: /'; 6 | # my @res = reportdata '{ suite_name => "SLBench-Benchmark-Runner" } :: //data//config_file'; 7 | 8 | <% to_json( unbless \@res ) %> 9 | %#<% Dumper($res[0]) %> 10 | EOTEMPLATE 11 | -------------------------------------------------------------------------------- /examples/dpath/warmup4_kvmpassthrough.mas: -------------------------------------------------------------------------------- 1 | #! mason debug=1 < "KVM-passthrough" } :: /'; 6 | % my @bogomips = reportdata '{ suite_name => "KVM-passthrough" } :: //description[value eq "- benchmark numbers"]/../_children//data'; 7 | %#<% Dumper(\@res) %> 8 | %#<% Dumper(\@bogomips) %> 9 | <% to_json( unbless \@bogomips ) %> 10 | EOTEMPLATE 11 | -------------------------------------------------------------------------------- /examples/dpath/xen_weekly.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | ### 4 | # Author : Conny Seidel 5 | # Descr. : plot "MetaReport: XEN testing effort" 6 | # Created: 2009-09-28 7 | # Updated: 2009-12-05 8 | ### 9 | ''' 10 | Plot timely overview of XEN testing effort 11 | ''' 12 | 13 | import re 14 | import simplejson 15 | import socket 16 | from matplotlib.pylab import * 17 | from matplotlib.colors import colorConverter 18 | 19 | Mason = '''#! mason <today(); 22 | % my $dt2 = DateTime->today()->subtract( weeks => 4); 23 | % my @res = reportdata "{suite_name => {'like', 'Topic-%'}, created_at => { -between => ['$dt2' , '$dt1' ] } } :: /"; 24 | % my @tests; 25 | % foreach my $lala (@res){ 26 | % my $platform = $lala ~~ dpath "//report/*/key_word/"; 27 | % my $cpu_type = $lala ~~ dpath "//revision"; 28 | % my $xen_keys = $lala ~~ dpath "//xen_version/../"; 29 | % if (keys %{$xen_keys->[0]} != 0){ 30 | % my $platf = "$platform->[0] ($cpu_type->[0])" || 'unknown'; 31 | % my $test = { "platform" => $platf, %{$xen_keys->[0]} }; 32 | % push @tests, $test; 33 | % } 34 | %} 35 | <% to_json( unbless \@tests) %> 36 | EOTEMPLATE 37 | ''' 38 | 39 | 40 | def pastel(colour, weight=2.4): 41 | ''' 42 | Convert colour into a nice pastel shade 43 | ''' 44 | 45 | rgb = asarray(colorConverter.to_rgb(colour)) 46 | # scale colour 47 | maxc = max(rgb) 48 | if maxc < 1.0 and maxc > 0: 49 | # scale colour 50 | scale = 1.0 / maxc 51 | rgb = rgb * scale 52 | # now decrease saturation 53 | total = sum(rgb) 54 | slack = 0 55 | for x in rgb: 56 | slack += 1.0 - x 57 | 58 | # want to increase weight from total to weight 59 | # pick x s.t. slack * x == weight - total 60 | # x = (weight - total) / slack 61 | x = (weight - total) / slack 62 | 63 | rgb = [c + (x * (1.0-c)) for c in rgb] 64 | 65 | return rgb 66 | 67 | def get_colours(n): 68 | ''' 69 | Return n pastel colours. 70 | ''' 71 | 72 | colours = [] 73 | for cols in range(n): 74 | list = [] 75 | for i in range(3): 76 | col = "%1.2f" % random() 77 | list.append(float(col)) 78 | colours.append(list) 79 | return [pastel(c) for c in colours[0:n]] 80 | 81 | 82 | def send_template(template): 83 | ''' 84 | send the template to the receiver api 85 | ''' 86 | 87 | s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 88 | received_data = '' 89 | try: 90 | s.connect(("165.204.15.71", 7358)) 91 | s.send(template) 92 | while 1: 93 | data = s.recv(4096) 94 | if not data: 95 | break 96 | received_data += data 97 | return_data = simplejson.loads(received_data) 98 | finally: 99 | s.close() 100 | return return_data 101 | 102 | 103 | def prep_data(data): 104 | ''' 105 | format, count and generate data structure 106 | ''' 107 | 108 | hardware = { '32 bit': {}, '64 bit': {} } 109 | version_list = [] 110 | plat_list = [] 111 | for i in data: 112 | # has "i686" in the value? 113 | if re.search(regex[0], i['xen_dom0_kernel']): 114 | bitness = '32 bit' 115 | # has "x86_64" in the value? 116 | elif re.search(regex[1], i['xen_dom0_kernel']): 117 | bitness = '64 bit' 118 | # short the expression for key 119 | version = i['xen_version'] 120 | # so that the plots are compareable we collect the versions 121 | if not version_list.count(version): 122 | version_list.append(version) 123 | # same shortcut here 124 | platf = i['platform'].upper().replace('_', '-') 125 | if not plat_list.count(platf): 126 | plat_list.append(platf) 127 | if not hardware[bitness].has_key(platf): 128 | hardware[bitness][platf] = {version: 1 } 129 | elif not hardware[bitness][platf].has_key(version): 130 | hardware[bitness][platf][version] = 1 131 | else: 132 | hardware[bitness][platf][version] += 1 133 | return hardware, sorted(version_list), plat_list 134 | 135 | 136 | regex = [ re.compile('i686'), re.compile('x86_64') ] 137 | test_data = send_template(Mason) 138 | results, versions, platforms = prep_data(test_data) 139 | platforms.sort() 140 | 141 | data = {'32 bit':[], '64 bit':[]} 142 | rows = {'32 bit':[], '64 bit':[]} 143 | for bit in results.keys(): 144 | for plat in platforms: 145 | tmp_data = [] 146 | # add data for missing platforms 147 | if not results[bit].has_key(plat): 148 | tmp_data.append([0] * len(versions)) 149 | else: 150 | for v in versions: 151 | if results[bit][plat].has_key(v): 152 | tmp_data.append(results[bit][plat][v]) 153 | else: 154 | tmp_data.append(0) 155 | data[bit].append(tmp_data) 156 | rows[bit].append(plat) 157 | ''' 158 | #debug 159 | print '%s\n%s\n%s' % (data, platforms, versions) 160 | ''' 161 | 162 | rc("font", family="terminus") 163 | rc("font", size=8) 164 | 165 | 166 | axes([0.2, 0.2, 0.7, 0.6]) # leave room below the axes for the table 167 | 168 | data1 = data['32 bit'] 169 | data2 = data['64 bit'] 170 | 171 | colLabels = versions 172 | rowLabels = platforms 173 | # Get some pastel shades for the colours 174 | colours = get_colours(len(rowLabels)) 175 | rows = len(data1) 176 | ind = arange(len(colLabels)) +0.3 # the x locations for the groups 177 | cellText1 = [] 178 | cellText2 = [] 179 | width = .45 # the width of the bars 180 | plt.subplots_adjust(bottom=0.2,left=0.23, right=0.90, hspace=0.85) 181 | 182 | plt.subplot(211) 183 | title('Weekly XEN testing effort per Platform') 184 | ylabel("32bit tests") 185 | yoff = array([0.0] * len(colLabels)) # the bottom values for stacked bar chart 186 | for row in xrange(rows): 187 | bar(ind, data1[row], width, bottom=yoff , color=colours[row]) 188 | yoff = yoff + data1[row] 189 | cellText1.append(['%i' % (x) for x in data1[row]]) 190 | cellText1.reverse() 191 | colours.reverse() 192 | rc("font", size=10) 193 | # Add a table at the bottom of the axes 194 | the_table = table(cellText=cellText1, 195 | rowLabels=rowLabels, rowColours=colours, 196 | colLabels=colLabels, 197 | loc='bottom') 198 | rc("font", size=8) 199 | xticks([]) 200 | colours.reverse() 201 | plt.subplot(212) 202 | ylabel("64bit tests") 203 | yoff = array([0.0] * len(colLabels)) # the bottom values for stacked bar chart 204 | for row in xrange(rows): 205 | bar(ind, data2[row], width, bottom=yoff, color=colours[row]) 206 | yoff = yoff + data2[row] 207 | cellText2.append(['%i' % (x) for x in data2[row]]) 208 | cellText2.reverse() 209 | colours.reverse() 210 | 211 | # Add a table at the bottom of the axes 212 | rc("font", size=10) 213 | the_table = table(cellText=cellText2, 214 | rowLabels=rowLabels, rowColours=colours, 215 | colLabels=colLabels, 216 | loc='bottom') 217 | 218 | xticks([]) 219 | savefig('xen_testing_effort', dpi=150) 220 | #show() 221 | 222 | # vim:set sr et ts=4 sw=4 st=4 ft=python syn=python fenc=utf-8: 223 | -------------------------------------------------------------------------------- /examples/how_things_work_together/README: -------------------------------------------------------------------------------- 1 | This is an example of test programs for Tapper together with everything 2 | needed for Tapper test automation. 3 | 4 | 5 | Files 6 | ===== 7 | 8 | bigfish.sh - test script 9 | bigfish_autoreport.sh - test script rewritten with autoreport 10 | bigfish.mpc - macro precondition to have the automation run the 11 | bigfish test 12 | report.mas - Mason file to search for bigfish with the 13 | report framework 14 | -------------------------------------------------------------------------------- /examples/how_things_work_together/bigfish.mpc: -------------------------------------------------------------------------------- 1 | # tapper-mandatory-fields: timeout,script 2 | # 3 | # This macro precodition installs a testrun with the bigfish test 4 | # program. It uses an image based approach for installation. The 5 | # alternatives are autoinstall and Cobbler. Both alternatives make 6 | # simpler preconditions but need more preparation up front. 7 | # 8 | # Preparation: 9 | # * put your image into $CONFIG->{paths}{image_dir} 10 | # * put the gen_initrd.sh into the directory given below 11 | # * put your kernel package into $CONFIG->{paths}{package_dir} 12 | # * put the bigfish test script to /data/tapper/packages/bigfish.sh 13 | # 14 | # Execute with 15 | # tapper-testrun new --macropreciondition bigfish.mpc --Dscript=/data/tapper/packages/bigfish.sh -Dtimeout=100 16 | # 17 | --- 18 | arch: linux64 19 | image: suse/suse_sles10_64b_smp_raw.tar.gz 20 | mount: / 21 | partition: testing 22 | precondition_type: image 23 | --- 24 | precondition_type: copyfile 25 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 26 | dest: /bin/ 27 | protocol: local 28 | --- 29 | precondition_type: package 30 | filename: kernel/x86_64/linux-2.6.32_rc6.x86_64.tgz 31 | --- 32 | precondition_type: exec 33 | filename: /bin/gen_initrd.sh 34 | options: 35 | - 2.6.32-rc6 36 | --- 37 | precondition_type: copyfile 38 | name: [% script %] 39 | dest: /bin/tapper-testscript 40 | protocol: nfs 41 | --- 42 | precondition_type: testprogram 43 | program: /bin/tapper-testscript 44 | timeout: [% timeout %] 45 | -------------------------------------------------------------------------------- /examples/how_things_work_together/bigfish.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # This is an example of the bigfish test suite. It does everything 4 | # needed for Tapper in shell. If you want to ease Tapper handling in 5 | # shell, have a look at autoreport. 6 | # 7 | # Bigfish reports a bogomips number for your system. This is not really 8 | # a valuable number but is great for demonstrating. 9 | # 10 | 11 | 12 | gen_output() 13 | { 14 | # tell Tapper how many tests to expect 15 | echo '1..3' 16 | 17 | # generate headers to provide additional information in Web GUI 18 | HOSTNAME=`hostname` 19 | echo "# Tapper-suite-name: BigFish 20 | # Tapper-suite-version: 1.0 21 | # Tapper-machine-name: $HOSTNAME 22 | # Tapper-section: BogoMIPS overview" 23 | 24 | # tell Web GUI which testrun this report belongs to 25 | if [ -n "$TAPPER_TESTRUN" ]; then 26 | echo "# Tapper-reportgroup-testrun: $TAPPER_TESTRUN" 27 | fi 28 | 29 | # check for logfile 30 | if [ ! -e /var/log/messages ]; then 31 | echo -n 'not ' 32 | fi 33 | echo 'ok - Logfile exists' 34 | 35 | # check whether logfile is readable 36 | if [ ! -r '/var/log/messages' ]; then 37 | echo -n 'not ' 38 | fi 39 | echo 'ok - Logfile is readable' 40 | 41 | MIPS_COUNT=$(grep -i 'bogomips' /var/log/dmesg |tail -1 | perl -e 'my $line = <>; my ($number) = $line =~ m/(\d+(\.\d+)?) BogoMIPS/; print $number;') 42 | 43 | if [ -z "$MIPS_COUNT" ] ; then 44 | echo -n 'not ' 45 | fi 46 | echo 'ok - BogoMIPS from dmesg' 47 | echo ' ---' 48 | echo " BogoMIPS: $MIPS_COUNT" 49 | echo ' ...' 50 | } 51 | 52 | if [ -n "$TAPPER_REPORT_SERVER" ];then 53 | gen_output | netcat $TAPPER_REPORT_SERVER $TAPPER_REPORT_PORT 54 | else 55 | gen_output 56 | fi 57 | -------------------------------------------------------------------------------- /examples/how_things_work_together/bigfish_autoreport.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # This is an example of the bigfish test suite using autoreport. It 4 | # shows how much easier test scripts can become using this feature. 5 | # 6 | # Bigfish reports a bogomips number for your system. This is not really 7 | # a valuable number but is great for demonstrating. 8 | # 9 | MIPS_COUNT=$(grep -i 'bogomips' /var/log/dmesg |tail -1 | perl -e 'my $line = <>; my ($number) = $line =~ m/(\d+(\.\d+)?) BogoMIPS/; print $number;') 10 | if [ -z "$MIPS_COUNT" ] ; then 11 | TAP[0]='no ok - BogoMIPS from dmesg' 12 | else 13 | TAP[0]='ok - BogoMIPS from dmesg' 14 | TAPDATA[0]="BogoMIPS:$MIPS_COUNT" 15 | fi 16 | 17 | source /data/tapper/autoreport/tapper-autoreport 18 | 19 | -------------------------------------------------------------------------------- /examples/how_things_work_together/bigfish_simple.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # tell Artemis how many tests to expect 4 | echo '1..3' 5 | 6 | # generate headers to provide additional information in Web GUI 7 | HOSTNAME=`hostname` 8 | echo "# Artemis-suite-name: BigFish 9 | # Artemis-suite-version: 1.0 10 | # Artemis-machine-name: $HOSTNAME 11 | # Artemis-section: BogoMIPS overview" 12 | 13 | # tell Web GUI which testrun this report belongs to 14 | if [ -n "$ARTEMIS_TESTRUN" ]; then 15 | echo "# Artemis-reportgroup-testrun: $ARTEMIS_TESTRUN" 16 | fi 17 | 18 | # check for logfile 19 | if [ ! -e /var/log/messages ]; then 20 | echo -n 'not ' 21 | fi 22 | echo 'ok - Logfile exists' 23 | 24 | # check whether logfile is readable 25 | if [ ! -r /var/log/messages ]; then 26 | echo -n 'not ' 27 | fi 28 | echo 'ok - Logfile is readable' 29 | 30 | MIPS_COUNT=`dmesg | grep 'Calibrating delay using timer specific routine'|tail -1 | perl -e 'my $line = <>; my ($number) = $line =~ m/(\d+(\.\d+)?) BogoMIPS/; print $number;'` 31 | 32 | if [ -z "$MIPS_COUNT" ] ; then 33 | echo -n 'not ' 34 | fi 35 | echo 'ok - BogoMIPS from dmesg' 36 | echo ' ---' 37 | echo " BogoMIPS: $MIPS_COUNT" 38 | echo ' ...' 39 | 40 | -------------------------------------------------------------------------------- /examples/how_things_work_together/report.mas: -------------------------------------------------------------------------------- 1 | #! mason < "BigFish" } :: /'; 5 | % use Data::Dumper; 6 | BogoMIPS found by bigfish tests: 7 | % foreach my $result (@results) { 8 | <% ($result ~~ dpath '//hostname')->[0] %>: <% ($result ~~ dpath '//BogoMIPS')->[0] %> 9 | % } 10 | EOTEMPLATE 11 | -------------------------------------------------------------------------------- /examples/preconditions/autoinstall.yml: -------------------------------------------------------------------------------- 1 | # Example for autoinstall precondition 2 | # 3 | # This example installs RHEL5 on the testmachine using kickstart. It 4 | # requires existing kickstartfiles accessible via HTTP. See section 5 | # webserver in chapter "Infrastructure" of the Tapper handbook on how to 6 | # prepare this web server. 7 | # 8 | # Note: The $TAPPER_OPTIONS part in the kernel command line is 9 | # required. Tapper substitutes this variable with everything the test 10 | # machine need to know to get its initial config. 11 | # 12 | # 13 | precondition_type: autoinstall 14 | name: opensuse-11.2-rc2-x86_64 15 | timeout: 10000 16 | grub_text: | 17 | timeout 2 18 | 19 | title RHEL 5 20 | kernel /tftpboot/stable/rhel/5/x86_64/vmlinuz console=ttyS0,115200 ks=http://tapper/autoinstall/stable/rhel/5/x86_64/tapper.ks ksdevice=eth0 noapic $TAPPER_OPTIONS 21 | initrd /tftpboot/stable/rhel/5/x86_64/initrd.img 22 | --- 23 | precondition_type: testprogram 24 | program: /opt/tapper/bin/py_parselog 25 | timeout: 600 26 | -------------------------------------------------------------------------------- /examples/preconditions/interdep.sc: -------------------------------------------------------------------------------- 1 | scenario_type: interdep 2 | description: 3 | - requested_hosts: 4 | - johnconnor 5 | preconditions: 6 | - arch: linux64 7 | image: suse/sles11_sp1_x86-64_baseimage.tar.gz 8 | mount: / 9 | partition: sda2 10 | precondition_type: image 11 | - precondition_type: copyfile 12 | name: /data/tapper/live/repository/testprograms/uname_tap/uname_tap.sh 13 | dest: /bin/ 14 | protocol: local 15 | - precondition_type: copyfile 16 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 17 | dest: /bin/ 18 | protocol: local 19 | - precondition_type: package 20 | filename: kernel/sles11/x86_64/linux-3.1.tgz 21 | - precondition_type: exec 22 | filename: /bin/gen_initrd.sh 23 | options: 24 | - 2.6.31-rc6 25 | - precondition_type: testprogram 26 | program: /opt/tapper/bin/tapper-testsuite-netperf-server 27 | timeout: 1000 28 | - requested_hosts: 29 | - sarahconnor 30 | preconditions: 31 | - arch: linux64 32 | image: suse/sles11_sp1_x86-64_baseimage.tar.gz 33 | mount: / 34 | partition: sda2 35 | precondition_type: image 36 | - precondition_type: copyfile 37 | name: /data/tapper/live/repository/testprograms/uname_tap/uname_tap.sh 38 | dest: /bin/ 39 | protocol: local 40 | - precondition_type: copyfile 41 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 42 | dest: /bin/ 43 | protocol: local 44 | - precondition_type: package 45 | filename: kernel/sles11/x86_64/linux-3.1_64.tgz 46 | - precondition_type: exec 47 | filename: /bin/gen_initrd.sh 48 | options: 49 | - 2.6.31-rc6 50 | - precondition_type: testprogram 51 | program: /opt/tapper/bin/tapper-testsuite-netperf-client 52 | timeout: 1000 53 | -------------------------------------------------------------------------------- /examples/preconditions/kernel_boot.mpc: -------------------------------------------------------------------------------- 1 | # tapper-mandatory-fields: kernel_version 2 | # 3 | # Example of a kernel boot macro precondition. 4 | # It demonstrates how to use Template Toolkit to make parts of the 5 | # precondition file configurable. In this case it allows to provide the 6 | # kernel during the tapper-testrun new command. 7 | # 8 | # Note: The "tapper-mandatory-fields" above is not part of the 9 | # comment. It is relevant for Tapper. It tells Tapper which fields have 10 | # to be given in the CLI call. 11 | --- 12 | arch: linux64 13 | image: suse/suse_sles10_64b_smp_raw.tar.gz 14 | mount: / 15 | partition: sda2 16 | precondition_type: image 17 | --- 18 | precondition_type: copyfile 19 | name: /data/tapper/live/repository/testprograms/uname_tap/uname_tap.sh 20 | dest: /bin/ 21 | protocol: local 22 | --- 23 | precondition_type: copyfile 24 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 25 | dest: /bin/ 26 | protocol: local 27 | --- 28 | [% kernelpkg = BLOCK %][% IF kernelpkg %][% kernelpkg %][%ELSE%]kernel/linux-[% kernel_version %].tar.gz[% END %][% END %] 29 | precondition_type: package 30 | filename: [% kernelpkg %] 31 | --- 32 | precondition_type: exec 33 | filename: /bin/gen_initrd.sh 34 | options: 35 | - [% kernel_version %] 36 | --- 37 | precondition_type: testprogram 38 | program: /opt/tapper/bin/tapper_testsuite/py_llc 39 | timeout: 72000 40 | 41 | # call with 42 | # tapper-testrun new --macroprecond ./kernel_boot.mpc -Dḱernel_version=3.1.0 43 | # OR 44 | # tapper-testrun new --macroprecond ./kernel_boot.mpc -Dḱernel_version=3.1.0 -Dkernelpkg = /opt/tapper/packages/kernel/linux-3.1.0.tar.gz -------------------------------------------------------------------------------- /examples/preconditions/kernel_boot_simple.mpc: -------------------------------------------------------------------------------- 1 | # Example of a kernel boot macro precondition. 2 | # It show you, how to have kernel_boot.mpc without using the Template 3 | # Toolkit functionality. This makes the precondition file easier but 4 | # less configurable. 5 | # 6 | --- 7 | arch: linux64 8 | image: suse/suse_sles10_64b_smp_raw.tar.gz 9 | mount: / 10 | partition: sda2 11 | precondition_type: image 12 | --- 13 | precondition_type: copyfile 14 | name: /data/tapper/live/repository/testprograms/uname_tap/uname_tap.sh 15 | dest: /bin/ 16 | protocol: local 17 | --- 18 | precondition_type: copyfile 19 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 20 | dest: /bin/ 21 | protocol: local 22 | --- 23 | precondition_type: package 24 | filename: kernel/x86_64/linux-2.6.31.x86_64.tgz 25 | --- 26 | precondition_type: exec 27 | filename: /bin/gen_initrd.sh 28 | options: 29 | - 2.6.31 30 | --- 31 | precondition_type: testprogram 32 | program: /opt/tapper/testprogram 33 | timeout: 72 34 | # call with 35 | # tapper-testrun new --macroprecond ./kernel_boot.mpc 36 | -------------------------------------------------------------------------------- /examples/preconditions/kernel_build.mpc: -------------------------------------------------------------------------------- 1 | --- 2 | arch: linux64 3 | image: suse/suse_sles10_sp2_64b_smp_raw.tar.gz 4 | mount: / 5 | partition: sda2 6 | precondition_type: image 7 | --- 8 | precondition_type: kernelbuild 9 | git_url: git://gitserver/linux-2.6.git 10 | changeset: HEAD 11 | --- 12 | precondition_type: testprogram 13 | program: /opt/tapper/bin/tapper_testsuite.sh 14 | timeout: 72000 15 | -------------------------------------------------------------------------------- /examples/preconditions/kernel_reboot.mpc: -------------------------------------------------------------------------------- 1 | # tapper-optional-fields: kernelpkg.file kernel_version 2 | --- 3 | arch: linux64 4 | image: suse/suse_sles10_64b_smp_raw.tar.gz 5 | mount: / 6 | partition: sda2 7 | precondition_type: image 8 | --- 9 | precondition_type: copyfile 10 | name: /data/tapper/live/repository/testprograms/uname_tap/uname_tap.sh 11 | dest: /bin/ 12 | protocol: local 13 | --- 14 | precondition_type: copyfile 15 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 16 | dest: /bin/ 17 | protocol: local 18 | --- 19 | [% IF kernelpkg %] 20 | precondition_type: exec 21 | filename: /bin/gen_initrd.sh 22 | options: 23 | - [% kernel_version %] 24 | --- 25 | precondition_type: package 26 | filename: [% kernelpkg %] 27 | [% ELSE %] 28 | precondition_type: produce 29 | producer: Kernel 30 | [% END %] 31 | --- 32 | precondition_type: testprogram 33 | program: /opt/tapper/bin/testprogam 34 | timeout: 300 35 | --- 36 | precondition_type: reboot 37 | count: 2 38 | -------------------------------------------------------------------------------- /examples/preconditions/kvm.yml: -------------------------------------------------------------------------------- 1 | name: automatically generated Xen test 2 | precondition_type: virt 3 | host: 4 | root: 5 | image: fedora/fedora-9-x86_64-latest.tar.bz2 6 | mount: / 7 | arch: linux64 8 | partition: sda2 9 | precondition_type: image 10 | preconditions: 11 | - type: git 12 | url: git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm.git 13 | precondition_type: repository 14 | - type: git 15 | url: git://git.kernel.org/pub/scm/virt/kvm/kvm-userspace.git 16 | precondition_type: repository 17 | - precondition_type: copyfile 18 | protocol: local 19 | name: /data/tapper/development/repository/configs/kvm/.config 20 | dest: /kvm/ 21 | - precondition_type: copyfile 22 | protocol: local 23 | name: /data/tapper/development/repository/packages/kvm/build_kvm.sh 24 | dest: /bin/ 25 | - precondition_type: exec 26 | filename: /bin/build_kvm.sh 27 | guests: 28 | - config: 29 | dest: /bin/startkvm.sh 30 | name: /data/tapper/live/repository/packages/tapperutils/startkvm.sh 31 | precondition_type: copyfile 32 | protocol: local 33 | exec: /bin/startkvm.sh 34 | root: 35 | dest: /kvm/images 36 | mountfile: /kvm/images/raw.img 37 | mounttype: raw 38 | name: fileserver:/export/image_files/official_testing/redhat_rhel5u2_64b_smp_up_qcow.img 39 | precondition_type: copyfile 40 | protocol: nfs 41 | arch: linux64 42 | testprogram: 43 | execname: /opt/tapper/bin/tapper_testsuite_kernbench.sh 44 | -------------------------------------------------------------------------------- /examples/preconditions/kvm_autoinstall.yml: -------------------------------------------------------------------------------- 1 | --- 2 | precondition_type: virt 3 | name: automatically generated KVM test 4 | host: 5 | root: 6 | grub_text: "timeout 2\n\ntitle RedHat Testing\nkernel /tftpboot/rhel/7.0/x86_64/vmlinuz ks=http://tapper/autoinstall/rhel/7.0/x86_64/kvm.ks ksdevice=link console=ttyS0,115200 $TAPPER_OPTIONS\ninitrd /tftpboot/rhel/7.0/x86_64/initrd.img\n" 7 | name: autoinstall-kvm-rhel-7 8 | precondition_type: autoinstall 9 | timeout: 10000 10 | testprogram_list: 11 | - 12 | execname: /opt/tapper/bin/metainfo 13 | runtime: 50 14 | timeout_testprogram: 300 15 | guests: 16 | - 17 | config: 18 | dest: /virt 19 | exec: /virt/guest1_startscript.sh 20 | name: tapper:/data/tapper/live/repository/configs/kvm/guest1_startscript.sh 21 | precondition_type: copyfile 22 | protocol: nfs 23 | root: 24 | arch: linux32 25 | mountfile: /virt/guest1_image.img 26 | mounttype: raw 27 | precondition_type: package 28 | target_directory: /virt 29 | url: nfs://fileserver:/export/image_files/official_testing/tared_testimages/opensuse_11_4_32b_raw.tgz 30 | testprogram_list: 31 | - 32 | execname: /opt/tapper/python/bin/py_ltp 33 | runtime: 28800 34 | timeout_testprogram: 36000 35 | - 36 | execname: /opt/tapper/bin/py_parselog 37 | runtime: 50 38 | timeout_testprogram: 200 39 | -------------------------------------------------------------------------------- /examples/preconditions/notification.yml: -------------------------------------------------------------------------------- 1 | persist: 0 2 | event: testrun_finished 3 | filter: testrun('id') == 42 4 | comment: Testrun id 42 finished 5 | -------------------------------------------------------------------------------- /examples/preconditions/oprofile.mpc: -------------------------------------------------------------------------------- 1 | # tapper-mandatory-fields: 2 | # tapper-optional-fields: id 3 | [% id = BLOCK %][% IF id %][% id %][%ELSE%]2009-06-29-perfmon-c4d4677_linux-v2.6.29-489-gd559a02_userland-5e7ac0e[% END %][% END %] 4 | --- 5 | arch: linux64 6 | image: suse/suse_sles10_sp2_64b_smp_raw.tar.gz 7 | mount: / 8 | partition: sda2 9 | precondition_type: image 10 | --- 11 | precondition_type: copyfile 12 | name: /data/tapper/live/repository/testprograms/uname_tap/uname_tap.sh 13 | dest: /bin/ 14 | protocol: local 15 | --- 16 | precondition_type: copyfile 17 | name: /data/tapper/live/repository/testprograms/oprofile/tapper-testsuite-oprofile 18 | dest: /bin/ 19 | protocol: local 20 | --- 21 | precondition_type: copyfile 22 | name: /data/tapper/live/repository/testprograms/oprofile/log2tap 23 | dest: /bin/ 24 | protocol: local 25 | --- 26 | precondition_type: copyfile 27 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 28 | dest: /bin/ 29 | protocol: local 30 | --- 31 | precondition_type: package 32 | filename: oprofile/perf-[% id %]-x86_64-standard.tar.gz 33 | --- 34 | precondition_type: package 35 | filename: oprofile/oprofile-[% id %]-x86_64.tar.gz 36 | --- 37 | precondition_type: package 38 | filename: oprofile/oprofile-[% id %]-testscripts.tar.gz 39 | --- 40 | precondition_type: testprogram 41 | program: /bin/tapper-testsuite-oprofile 42 | timeout: 1200 43 | --- 44 | precondition_type: testprogram 45 | program: /opt/tapper/bin/py_parselog 46 | timeout: 120 47 | --- 48 | # precondition_type: installer_stop 49 | 50 | # use this with: 51 | # tapper-testrun new --macroprecond ~/tmp/oprofile.mpc -Did=2009-06-29-perfmon-c4d4677_linux-v2.6.29-489-gd559a02_userland-5e7ac0e --host johnconnor --owner root 52 | 53 | -------------------------------------------------------------------------------- /examples/preconditions/precondition_for_discover.yml: -------------------------------------------------------------------------------- 1 | --- 2 | arch: linux64 3 | image: suse/sles11_sp1_x86-64_baseimage.tar.gz 4 | mount: / 5 | partition: 6 | - testing 7 | - sda2 8 | - hda2 9 | precondition_type: image 10 | --- 11 | dest: /bin/ 12 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 13 | precondition_type: copyfile 14 | protocol: local 15 | --- 16 | dest: /bin/ 17 | name: /data/tapper/live/repository/testprograms/autoreport/tapper-run-autoreport 18 | precondition_type: copyfile 19 | protocol: local 20 | --- 21 | filename: kernel/sles11/x86_64/stable/linux-stable-3.3.2012-03-31.x86_64.tgz 22 | precondition_type: package 23 | --- 24 | ## Changed to packages compared to upstream: 25 | ## * cd $(dirname $0) as first line in Install-AMD-APP.sh, so we don't need to call it from current dir 26 | filename: opencl/amd-sdk/AMD-APP-SDK-v2.6-lnx64.tgz 27 | precondition_type: package 28 | target_directory: /tmp/opencl/ 29 | --- 30 | filename: /tmp/opencl/Install-AMD-APP.sh 31 | precondition_type: exec 32 | --- 33 | filename: /bin/gen_initrd.sh 34 | options: 35 | - 3.3.0-c16fa4f2-stable 36 | precondition_type: exec 37 | --- 38 | dest: /etc/ 39 | name: /etc/resolv.conf 40 | precondition_type: copyfile 41 | protocol: local 42 | --- 43 | filename: /opt/tapper/perl/perls/current/bin/cpan 44 | options: 45 | - OpenCL 46 | environment: 47 | PASTHRU_INC: '-I/opt/AMDAPP/include/' 48 | precondition_type: exec 49 | --- 50 | config: "serial --unit=0 --speed=115200\nterminal serial\ntimeout 3\ndefault 0\ntitle osrc kernel 3.3.0-c16fa4f2-stable\n root $grubroot\n kernel /boot/vmlinuz root=$root console=ttyS0,115200 earlyprintk=ttyS0,115200 debug ignore_loglevel\n initrd /boot/initrd\n" 51 | precondition_type: grub 52 | --- 53 | precondition_type: installer_stop 54 | -------------------------------------------------------------------------------- /examples/preconditions/simulator.yml: -------------------------------------------------------------------------------- 1 | --- 2 | precondition_type: hint 3 | arch: linux64 4 | ssh: 1 5 | --- 6 | precondition_type: copyfile 7 | dest: /tmp/images/ubuntu_image_new.hdd 8 | name: fileserver:/export/image_files/official_testing/ubuntu_image_new.hdd 9 | protocol: nfs 10 | --- 11 | precondition_type: package 12 | filename: tapperutils/opt-tapper64.tar.gz 13 | mountfile: /tmp/images/ubuntu_image_new.hdd 14 | mountpartition: p1 15 | --- 16 | precondition_type: prc 17 | config: 18 | guest_number: 0 19 | testprogram_list: 20 | - 21 | execname: /opt/tapper/python/bin/metainfo 22 | timeout_testprogram: 2000 23 | runtime: 1000 24 | - 25 | execname: /opt/tapper/bin/py_parselog 26 | runtime: 50 27 | timeout_testprogram: 200 28 | mountfile: /tmp/images/ubuntu_image_new.hdd 29 | mountpartition: p1 30 | --- 31 | precondition_type: testprogram 32 | program: /bin/tapper_start_simulator.sh 33 | parameters: 34 | - -r 1 35 | - -v 36 | timeout: 4800 37 | -------------------------------------------------------------------------------- /examples/preconditions/suspend_resume.yml: -------------------------------------------------------------------------------- 1 | --- 2 | arch: linux32 3 | image: suse/sles11_sp1_i686_baseimage.tgz 4 | mount: / 5 | partition: 6 | - testing 7 | - sda2 8 | - hda2 9 | precondition_type: image 10 | --- 11 | dest: /bin/ 12 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 13 | precondition_type: copyfile 14 | protocol: local 15 | --- 16 | filename: /data/tapper/live/repository/packages/testsuites/misc/resume_benchmark.tgz 17 | precondition_type: package 18 | --- 19 | filename: kernel/sles11/i686/stable/linux-stable-3.3.6.2012-05-12.i686.tgz 20 | precondition_type: package 21 | --- 22 | filename: /bin/gen_initrd.sh 23 | options: 24 | - 3.3.6-b67be2a5-stable 25 | precondition_type: exec 26 | --- 27 | precondition_type: testprogram 28 | program: /bin/resume_benchmark.pl 29 | timeout: 300 30 | capture: tap 31 | environment: 32 | BENCHMARK_HOSTNAME: tapper:2000 33 | -------------------------------------------------------------------------------- /examples/preconditions/test_env.mpc: -------------------------------------------------------------------------------- 1 | --- 2 | arch: linux64 3 | image: suse/suse_sles10_64b_smp_raw.tar.gz 4 | mount: / 5 | partition: sda2 6 | precondition_type: image 7 | --- 8 | precondition_type: copyfile 9 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 10 | dest: /bin/ 11 | protocol: local 12 | --- 13 | precondition_type: package 14 | filename: kernel/x86_64/linux-2.6.31.x86_64.tgz 15 | --- 16 | precondition_type: exec 17 | filename: /bin/gen_initrd.sh 18 | options: 19 | - 2.6.31-rc6 20 | --- 21 | precondition_type: copyfile 22 | name: /data/tapper/live/repository/testprograms/test_env/test_env 23 | dest: /bin/ 24 | protocol: local 25 | --- 26 | precondition_type: testprogram 27 | program: /bin/test_env 28 | timeout: 720 29 | runtime: 100 30 | parameters: 31 | - -l 32 | - -v 33 | -------------------------------------------------------------------------------- /examples/preconditions/testplan.mpc: -------------------------------------------------------------------------------- 1 | --- 2 | type: multitest 3 | description: 4 | shortname: generic_test on Tip kernel/ 64bit 5 | requested_hosts_all: 6 | - johnconnor 7 | preconditions: 8 | - arch: linux64 9 | image: suse/sles11_sp1_x86-64_baseimage.tar.gz 10 | mount: / 11 | partition: 12 | - testing 13 | - sda2 14 | - hda2 15 | precondition_type: image 16 | - dest: /bin/ 17 | name: /data/tapper/live/repository/packages/kernel/gen_initrd.sh 18 | precondition_type: copyfile 19 | protocol: local 20 | - filename: kernel/sles11/osrc/linux-tip-2.6.38.x86_64.tgz 21 | precondition_type: package 22 | - filename: /bin/gen_initrd.sh 23 | options: 24 | - 2.6.38 25 | precondition_type: exec 26 | - precondition_type: copyfile 27 | name: /data/tapper/autoreport/generic_test.sh 28 | dest: / 29 | protocol: local 30 | - precondition_type: testprogram 31 | program: /generic_test.sh 32 | timeout: 300 # 5 minutes 33 | --- 34 | type: multitest 35 | description: 36 | shortname: generic_test on Tip kernel/ 32bit 37 | requested_hosts_all: 38 | - johnconnor 39 | preconditions: 40 | - arch: linux64 41 | image: suse/sles11_sp1_x86-64_baseimage.tar.gz 42 | mount: / 43 | partition: 44 | - testing 45 | - sda2 46 | - hda2 47 | precondition_type: image 48 | - dest: /bin/ 49 | name: /data/tapper/live/repository/packages/kernel/gen_initrd.sh 50 | precondition_type: copyfile 51 | protocol: local 52 | - filename: kernel/sles11/osrc/linux-tip-2.6.38.i686.tgz 53 | precondition_type: package 54 | - filename: /bin/gen_initrd.sh 55 | precondition_type: exec 56 | options: 57 | - 2.6.38 58 | - precondition_type: copyfile 59 | name: /data/tapper/autoreport/generic_test.sh 60 | dest: / 61 | protocol: local 62 | - precondition_type: testprogram 63 | program: /generic_test.sh 64 | timeout: 300 # 5 minutes 65 | -------------------------------------------------------------------------------- /examples/preconditions/xen_temare.yml: -------------------------------------------------------------------------------- 1 | name: automatically generated Xen test 2 | precondition_type: virt 3 | host: 4 | root: 5 | arch: linux64 6 | image: suse/suse_sles10_sp2_64b_smp_raw.tar.gz 7 | mount: / 8 | partition: 9 | - sda2 10 | - testing 11 | - hda2 12 | precondition_type: image 13 | preconditions: 14 | - filename: /data/tapper/live/repository/packages/xen/builds/x86_64/xen-unstable/xen-unstable.x86_64.tgz 15 | precondition_type: package 16 | - filename: tapperutils/sles10/xen_installer_suse.tar.gz 17 | precondition_type: package 18 | - filename: /bin/xen_installer_suse.pl 19 | precondition_type: exec 20 | - line: fileserver:/home /home nfs auto,defaults 0 0 21 | precondition_type: fstab 22 | guests: 23 | - config: 24 | dest: /xen/images/ 25 | filename: /xen/images/guest1_xenconfig.svm 26 | name: tapper:/data/tapper/live/repository/configs/xen/guest1_xenconfig.svm 27 | precondition_type: copyfile 28 | protocol: nfs 29 | root: 30 | dest: /xen/images/ 31 | name: fileserver:/export/image_files/official_testing/redhat_rhel5u5_64b_qcow.img 32 | precondition_type: copyfile 33 | protocol: nfs 34 | arch: linux64 35 | mounttype: raw 36 | mountfile: /xen/images/guest1_image.img 37 | testprogram: 38 | execname: /opt/tapper/bin/tapper_testsuite_parselogs 39 | timeout_testprogram: 10800 40 | runtime: 3600 41 | --- 42 | precondition_type: grub 43 | shortname: grub 44 | config: | 45 | serial --unit=0 --speed=115200 46 | terminal serial 47 | timeout 3 48 | default 0 49 | 50 | title XEN 51 | root $grubroot 52 | kernel /boot/xen.gz console=com1 com1=115200 watchdog nmi=fatal loglvl=all guest_loglvl=all noreboot 53 | module /boot/vmlinuz-2.6.18.8-xen root=$root console=ttyS0,115200 xencons=ttyS 54 | module /boot/initrd-2.6.18.8-xen 55 | 56 | -------------------------------------------------------------------------------- /examples/preconditions/xen_windows.yml: -------------------------------------------------------------------------------- 1 | name: automatically generated Xen test 2 | precondition_type: virt 3 | host: 4 | root: 5 | arch: linux64 6 | image: suse/suse_sles10_sp2_64b_smp_raw.tar.gz 7 | mount: / 8 | partition: 9 | - sda2 10 | - testing 11 | precondition_type: image 12 | preconditions: 13 | - filename: /data/tapper/live/repository/packages/xen/builds/x86_64/xen-3.3-testing/xen-3.3-testing.x86_64.tgz 14 | precondition_type: package 15 | - filename: tapperutils/sles10/xen_installer_suse.tar.gz 16 | precondition_type: package 17 | - filename: /bin/xen_installer_suse.pl 18 | precondition_type: exec 19 | guests: 20 | - config: 21 | dest: / 22 | svm: /guest1_config_windows.svm 23 | name: tapper:/data/tapper/tapper/live/repository/configs/xen/guest1_config_windows.svm 24 | precondition_type: copyfile 25 | protocol: nfs 26 | root: 27 | arch: windows 28 | dest: /xen/images/ 29 | mountfile: /xen/images/guest1_windows.img 30 | mountpartition: p1 31 | mounttype: windows 32 | name: fileserver:/export/image_files/official_testing/ms_win2003_sp2_32b_raw.img 33 | precondition_type: copyfile 34 | protocol: nfs 35 | testprogram: 36 | execname: winsst 37 | timeout_testprogram: 100 38 | runtime: 10 39 | -------------------------------------------------------------------------------- /examples/testplans/include/defaultbenchmarks: -------------------------------------------------------------------------------- 1 | [%#- -*- mode: tt -*- %] 2 | [% FILTER collapse %] 3 | [% IF ourbenchmark.match('^aim|autotest|ctcs|diy|hackbench|hostcputest|hts|kernbench|llc|lmbench|ltp|perf|phoronix|rhv7|slbench|speccpu2006|specjbb2005|stressapptest|sysbench|tiobench|usercompbench$') %] 4 | [% benchmarkscript = BLOCK %]/opt/tapper/bin/tapper_testsuite_[% ourbenchmark %].sh[% END %] 5 | [% ELSE %] 6 | [% THROW UNKNOWNBENCHMARK "Unknown benchmark name.\n" %] 7 | [% END %] 8 | [% END %][%# FILTER collapse %] 9 | -------------------------------------------------------------------------------- /examples/testplans/include/defaults: -------------------------------------------------------------------------------- 1 | [%#- -*- mode: tt -*- %] 2 | [%- current_SLES11 = 'suse/sles11_sp1_x86-64_baseimage.tar.gz' %] 3 | [%- SLES_11_autoinstall_version = '11.1' %] 4 | [%- current_linux_3_0_package = 'kernel/sles11/x86_64/linux-3.1_rc3-96-gcaca951.2011-08-25.x86_64.tgz' %] 5 | [%- autotest_snapshot_url = 'file:///data/tapper/live/repository/packages/autotest/renormalist-autotest-039effb.tar.gz' %] 6 | [%- Xen_machines = ['limete','arges', 'gawaine'] %] 7 | [%- builder_list = [ '40', '40pv', '41', '41pv', '4u', '4us', '4ubd', '4upv', '4uspv', 'osvw', '41vmcb', '4uvmcb', '4udecode', '34', '33', '32', '31' ] %] 8 | [%- kernel_builder_list = [ 'linus', 'tip', 'stable', 'stable2-*', 'stable3-*', 'longterm-*' ] %] 9 | [%- machine_list = [ 'foo', 'bar', 'baz' ] %] 10 | [%- all_autotest_tests = ['qemu_iotests', 'iperf', 'linus_stress', 11 | 'iosched_bugs', 'fsstress', 'sparse', 'sysbench', 12 | 'dbench', 'fsfuzzer', 'ipv6connect', 'stress', 13 | 'fio', 'libhugetlbfs', 'kvm', 'fsx', 'selftest', 14 | 'bash_shared_mapping', 'dbt2', 'signaltest', 15 | 'isic', 'barriertest', 'btreplay', 'systemtap', 16 | 'tiobench', 'iozone', 'fsdev', 'cerberus', 17 | 'reaim', 'profiler_test', 'pi_tests', 'rmaptest', 18 | 'hwclock', 'bonnie', 'error_test_fail', 'dacapo', 19 | 'cpu_hotplug', 'spew', 'perfmon', 'npb', 20 | 'error_test_na', 'error_initialize', 21 | 'error_skip_step', 'error_test_bug', 22 | 'monotonic_time', 'error_cleanup', 'unixbench5', 23 | 'cyclictest', 'netpipe', 'aiostress', 24 | 'compilebench', 'real_time_tests', 'lmbench', 25 | 'disktest', 'memory_api', 'synctest', 26 | 'unixbench', 'pktgen', 'hackbench', 27 | 'parallel_dd', 'posixtest', 'ltp', 'interbench', 28 | 'error_test_error', 'ebizzy', 'download', 29 | 'fs_mark', 'kvmtest', 'tbench', 'aborttest', 30 | 'aio_dio_bugs', 'kernbench', 'uptime', 31 | 'kernelbuild', 'dma_memtest', 'wb_kupdate', 32 | 'tsc', 'scrashme', 'netperf2', 'profiler_sync', 33 | 'flail', 'tracing_microbenchmark', 'lsb_dtk', 34 | 'sleeptest', 'rtc', 'xmtest', 'rttester', 35 | 'error_setup'] %] 36 | [%- useful_autotest_tests = [ 'hackbench', 'sleeptest', 'unixbench', 37 | 'tbench', 'dbench', 'tiobench', 38 | 'compilebench', 'lmbench', 'iozone', 39 | 'kernbench', 'reaim', 'aiostress', 40 | 'bonnie', 'compilebench', 'linus_stress', 41 | 'fsstress', 'ltp', 'stress' ] %] 42 | [%- unused_autotest_tests = [ 'sysbench', 'dacapo', 'unixbench5' ] %][%# --- dysfunctional --- %] 43 | [%- distro_list = ['rhel_6.2_64', 44 | 'fedora_15_64', 45 | 'sles_11.2_64', 46 | 'opensuse_12.1_64', 47 | 'ubuntu_11.10_64', 48 | ] %] 49 | [%- image_list = [ 50 | 'suse_sles11_sp2_64b_raw.img', 51 | 'suse_sles11_sp2_32b_raw.img', 52 | 'suse_sles11_sp2_32bpae_raw.img', 53 | 'redhat_rhel6u2_64b_raw.img', 54 | 'redhat_rhel6u2_32b_raw.img', 55 | 'redhat_rhel5u8_64b_raw.img', 56 | 'redhat_rhel5u8_32b_raw.img', 57 | 'ms_winxp_pro_sp3_32b_sysprep_raw.img', 58 | 'ms_winxp_pro_sp2_64b_sysprep_raw.img', 59 | 'ms_windows_2003_sp2_64b_sysprep_raw.img', 60 | 'ms_windows_2003_sp2_32b_sysprep_raw.img', 61 | 'ms_windows_2008_32b_sysprep_raw.img', 62 | 'ms_windows_2008_64b_sysprep_raw.img', 63 | ] %] 64 | 65 | -------------------------------------------------------------------------------- /examples/testplans/include/distrodetails: -------------------------------------------------------------------------------- 1 | [%#- -*- mode: tt -*- %] 2 | [% FILTER collapse %] 3 | [% IF distro.match('rhel_6.3_64') %] 4 | [% install_file = 'ks=http://tapper/autoinstall/rhel/6.3/x86_64/bare.ks' %] 5 | [% install_repo = 'repo=ftp://fileserver/rhel/6.3/x86_64/os' %] 6 | [% install_opts = 'ksdevice=link' %] 7 | [% kernel = '/tftpboot/autoinstall/rhel/6.3/x86_64/vmlinuz' %] 8 | [% initrd = '/tftpboot/autoinstall/rhel/6.3/x86_64/initrd.img' %] 9 | [% END %] 10 | [% IF distro.match('sles_11.2_64') %] 11 | [% install_file = 'autoyast=http://tapper/autoinstall/sles/11.2/x86_64/bare.xml' %] 12 | [% install_repo = 'install=ftp://fileserver/sles/11.2/x86_64' %] 13 | [% install_opts = 'textmode=1' %] 14 | [% kernel = '/tftpboot/sles/11.2/x86_64/linux' %] 15 | [% initrd = '/tftpboot/sles/11.2/x86_64/initrd' %] 16 | [% END %] 17 | [% IF distro.match('ubuntu_12.04_64') %] 18 | [% install_file = 'url=http://tapper/autoinstall/ubuntu/12.04/x86_64/preseed' %] 19 | [% install_repo = '' %] 20 | [% install_opts = 'hostname=ubuntu locale=en_US.UTF-8 keyboard-configuration/layout=USA keyboard-configuration/variant=USA console-setup/ask_detect=false console=ttyS0,115200 interface=eth0' %] 21 | [% kernel = '/tftpboot/ubuntu/12.04/x86_64/linux' %] 22 | [% initrd = '/tftpboot/ubuntu/12.04/x86_64/initrd.gz' %] 23 | [% END %] 24 | [% IF distro.match('fedora_15_64') %][%# =============================================== TODO: REVIEW AND FIX === %] 25 | [% install_file = 'ks=http://tapper/autoinstall/fedora/15/x86_64/bare.ks' %] 26 | [% install_repo = 'repo=ftp://fileserver/fedora/releases/15/Fedora/x86_64/os' %] 27 | [% install_opts = 'ksdevice=link' %] 28 | [% kernel = '/tftpboot/autoinstall/fedora/15/x86_64/vmlinuz' %] 29 | [% initrd = '/tftpboot/autoinstall/fedora/15/x86_64/initrd.img' %] 30 | [% END %] 31 | [% IF distro.match('opensuse_12.1_64') %][%# =============================================== TODO: REVIEW AND FIX === %] 32 | [% install_file = 'autoyast=http://tapper/autoinstall/opensuse/12.1/x86_64/bare.xml' %] 33 | [% install_repo = 'install=ftp://fileserver/opensuse/distribution/12.1/repo/oss' %] 34 | [% install_opts = 'textmode=1' %] 35 | [% kernel = '/tftpboot/autoinstall/opensuse/12.1/x86_64/linux' %] 36 | [% initrd = '/tftpboot/autoinstall/opensuse/12.1/x86_64/initrd' %] 37 | [% END %] 38 | [% IF distro.match('-xen') %] 39 | [% install_file = install_file.replace('bare.xml', 'xen.xml') %] 40 | [%# no xen.ks because no Xen in RHEL %] 41 | [% END %] 42 | [% IF distro.match('-kvm') %] 43 | [% install_file = install_file.replace('bare.ks', 'kvm.ks').replace('bare.xml', 'kvm.xml') %] 44 | [% END %] 45 | [% END %][%# FILTER collapse %] 46 | -------------------------------------------------------------------------------- /examples/testplans/kernel/benchmark: -------------------------------------------------------------------------------- 1 | [%#- -*- mode: tt -*- %] 2 | 3 | [%- IF title == '' %] 4 | [%- title = "Kernel Performance Regression" -%] 5 | [% END -%] 6 | 7 | [%- IF machines == '' %] 8 | [%- machines = ['foo', 'bar', 'baz' ] -%] 9 | [% END -%] 10 | 11 | [%- IF benchmarks == '' %] 12 | [%- benchmarks = ['kernbench', 'hackbench', 'sysbench', 'lmbench', 'usercompbench', 'specjbb2005', 'specjvm2008'] -%] 13 | [% END -%] 14 | 15 | [%- IF kernelpkg == '' %] 16 | [% THROW kernelpkg "Missing kernelpkg. You have to provide it." %] 17 | [% END -%] 18 | 19 | [%- IF gitrev == '' %] 20 | [% THROW gitrev "Missing 'gitrev'. You have to provide it." %] 21 | [% END -%] 22 | 23 | [% gitconfig = 'kernel_rev-' _ gitrev _ '.log' %] 24 | [% perf = 'perf-' _ gitrev %] 25 | 26 | [%- IF queue == '' %] 27 | [%- queue = "Kernel-benchmarks" -%] 28 | [% END -%] 29 | 30 | [%- IF topic == '' %] 31 | [%- topic = "Test-Kernel-Performance-Regression" -%] 32 | [% END -%] 33 | 34 | [%- IF kernelpkg_path == '' %] 35 | [%- kernelpkg_path = "/data/tapper/live/repository/packages/kernel/slbench/" -%] 36 | [% END -%] 37 | 38 | [%- IF benchmark_path == '' %] 39 | [%- benchmark_path = "/opt/tapper/python/bin/" -%] 40 | [% END -%] 41 | 42 | [% FOREACH benchmark IN benchmarks %] 43 | [%- PROCESS 'topic/kernel/generic/benchmark' -%] 44 | [% END %] 45 | 46 | 47 | -------------------------------------------------------------------------------- /examples/testplans/kernel/generic/benchmark: -------------------------------------------------------------------------------- 1 | ### 2 | ### 3 | ### 4 | ### Name : Linux Kernel Performance Regression Benchmarking 5 | ### Host system : Ubuntu 10.04 6 | ### Kernel : Linus tree 7 | ### Test program: to be filled in 8 | ### 9 | ### 10 | ### 11 | --- 12 | type: multitest 13 | description: 14 | shortname: "[% title %] - [% kernelpkg %]" 15 | topic: [% benchmark %] 16 | queue: [% queue %] 17 | requested_hosts_all: 18 | [% FOREACH machine IN machines %] - [% machine %] 19 | [% END %] 20 | preconditions: 21 | - 22 | precondition_type: image 23 | arch: linux64 24 | image: ubuntu/lkbench-64.tar.gz 25 | mount: / 26 | partition: 27 | - testing 28 | - sda2 29 | - hda2 30 | - 31 | precondition_type: copyfile 32 | dest: /usr/local/bin/ 33 | name: /data/tapper/live/repository/packages/artemisutils-legacy/kernel/install_debian_kernel.sh 34 | precondition_type: copyfile 35 | protocol: local 36 | - 37 | precondition_type: copyfile 38 | dest: /usr/src/ 39 | name: [% kernelpkg_path %][% kernelpkg %] 40 | precondition_type: copyfile 41 | protocol: local 42 | - 43 | precondition_type: copyfile 44 | dest: /usr/local/bin/ 45 | name: [% kernelpkg_path %][% perf %] 46 | protocol: local 47 | - 48 | precondition_type: copyfile 49 | dest: /etc/ 50 | name: [% kernelpkg_path %][% gitconfig %] 51 | protocol: local 52 | - 53 | precondition_type: exec 54 | filename: /usr/local/bin/install_debian_kernel.sh 55 | - 56 | precondition_type: testprogram 57 | program: /opt/tapper/bin/metainfo 58 | timeout: 120 59 | - 60 | precondition_type: testprogram 61 | program: [% benchmark_path %]tapper_testsuite_[% benchmark %].sh 62 | parameters: 63 | - --speedcenter-upload 64 | runtime: 86400 65 | timeout: 90000 66 | - 67 | precondition_type: grub 68 | config: "serial --unit=0 --speed=115200\nterminal serial\ntimeout 3\ndefault 0\ntitle lk-speed-regression\n root $grubroot\n kernel /boot/vmlinuz root=$root console=ttyS0,115200 reboot=force\n initrd /boot/initrd.img\n" 69 | -------------------------------------------------------------------------------- /examples/testplans/kernel/generic/test: -------------------------------------------------------------------------------- 1 | [%#- -*- mode: tt -*- %] 2 | [%- PROCESS 'include/defaults' -%] 3 | [% USE Autoformat %] 4 | [%- IF machine != '' %][% THROW LEGACY "Use '-Dmachines=...' (plural) now.\n" %][% END -%] 5 | [%- IF tests_dom0 != '' %][% THROW LEGACY "Use '-Dtest_host=...' now.\n" %][% END -%] 6 | [%- IF tests_domu != '' %][% THROW LEGACY "Use '-Dtests_guest=...' now.\n" %][% END -%] 7 | [%- IF Timeout != '' %][% THROW LEGACY "Use '-Dtest_timout=...' now.\n" %][% END -%] 8 | [%- IF tests != '' %][% THROW AMBIGUITY "Use '-Dtests_host=...'.\n" %][% END -%] 9 | [%- IF host != '' %][% THROW AMBIGUITY "Use '-Dmachines=...'.\n" %][% END -%] 10 | [%- IF hosts != '' %][% THROW AMBIGUITY "Use '-Dmachines=...'.\n" %][% END -%] 11 | [%- IF queues != '' %][% THROW AMBIGUITY "Use '-Dqueue=name' (singular).\n" %][% END -%] 12 | [%- IF queue.match(',') %][% THROW AMBIGUITY "Use '-Dqueue=name' (only one queue).\n" %][% END -%] 13 | [%- IF queue == '' %][% queue = 'AdHoc' %][% END -%] 14 | [%- IF machines == '' %][% machines = 'limete' %][% END -%] 15 | [%- IF distros == '' %][% distros = 'baseimage' %][% END -%] 16 | [%- IF buildserver == '' %][% buildserver = 'acid' %][% END -%][%# acid=64bit, selen=32bit %] 17 | [%- IF packages_host == '' %][% packages_host = '' %][% END -%] 18 | [%- IF tests_host == '' %][% tests_host = 'linus_stress' %][% END -%][%# 'hackbench,unixbench,dbench,tiobench,iozone,reaim,aiostress,compilebench' %] 19 | [%- IF test_timeout == '' %][% test_timeout = '10800' %][% END -%] 20 | [%- shorttitle_test_host = BLOCK %][% IF tests_host.match(',') %]MULTI[% ELSE %][% tests_host.replace('.*/([^/]+)$', '$1').replace('^(xen|kvm)-','').replace('\.sh$','').replace('\.t$','').replace('[-_]','') %][% END %][% END -%] 21 | [%- IF title == '' %][% title = BLOCK %]Kernel[%- IF shorttitle_test_host != '' %]-[% shorttitle_test_host %][% ELSE %][% END %][% END %][% END -%] 22 | [%- IF builder == '' && distros == 'baseimage' && kernelpkg == '' %][% builder = 'linus' %][% END -%] 23 | [% repository = builder %] 24 | [% type = 'kernel' %] 25 | [%- AllMachines = machines.split(',') || [] -%] 26 | [%- AllDistros = distros.split(',') || [] -%] 27 | [%- AllPackagesHost = packages_host.split(',') || [] -%] 28 | [%- AllTestsHost = tests_host.split(',') || [] -%] 29 | [%- AllTestsParamsHost = tests_host_params.split(';') || [] -%] 30 | [%- AllHostBootparams = hostbootparams.split(';') || [] -%][%# SIC, semicolon! We need ',' inside bootparams. -%] 31 | [% fullname = BLOCK %][% title %][% IF titlesuffix %]-[% titlesuffix %][% END %][% END %] 32 | [%- IF AllHostBootparams.size == 0 %][%- AllHostBootparams.unshift('') %][% END -%][%# to not empty the loop over it -%] 33 | ### 34 | ### Run a Linux environment with tests. 35 | ### [topic.kernel.generic.test] 36 | ### 37 | ### 38 | ### Name : [% fullname %] 39 | ### Host : [% current_SLES11 %] 40 | ### 41 | ### Optional params: 42 | ### 43 | ### -Dkernelpkg= Filename of already existing kernel package; default: build one 44 | ### -Dbuilder= Key what type of kernel to build; default: stable3-2 45 | ### -Dbuilder_changeset= Changeset of kernel to build; default: 46 | ### 47 | ### -Dhostbootparams= Kernel boot parameter to host, SEMICOLON[sic] separated; default: 48 | ### -Dtest_timeout= Timeout for tests; default: 10800 (3h) 49 | ### 50 | ### -Dtests_host= Workload names, comma separated, see list below; default: 51 | ### -Dtests_host_capture=tap Capture and send TAP output of host tests; default: 52 | ### -Dpackages_host= Files to unpack in host, comma separated; default: 53 | ### 54 | ### -Ddistros= Host distro names, comma separated; see list below; default: sles_11.2_64 55 | 56 | ### -Dmachines= Machine names, comma separated; default: limete 57 | ### -Dqueue= Queue to use; default: AdHoc 58 | ### -Dtitlesuffix= Suffix to be used in testplan and testrun names; default: 59 | ### 60 | ### Available values: 61 | ### 62 | ### builder: 63 | [%- FILTER autoformat(all => 1) %] 64 | ### [% FOREACH m = kernel_builder_list -%][% m %], [% END %] 65 | [% END %] 66 | ### Mnemonics: 67 | ### - "*" means "use the minor wanted number" 68 | ### machines: 69 | [%- FILTER autoformat(all => 1) %] 70 | ### [% FOREACH m = machine_list -%][% m %], [% END %] 71 | [% END %] 72 | ### distros: 73 | [%- FILTER autoformat(all => 1) %] 74 | ### [% FOREACH d = distro_list -%][% d %], [% END %] 75 | [% END %] 76 | ### tests_host: 77 | ### - non-path words are interpreted as Autotest subtests: 78 | ### 79 | [%- FILTER autoformat(all => 1) %] 80 | ### [% FOREACH t = useful_autotest_tests -%][% t %], [% END %] 81 | [%- END -%] 82 | ### 83 | ### - absolute path names, like "/data/tapper/autoreport/mytest.t" 84 | ### are interpreted as script to execute (and every file inside 85 | ### that path/* will also be copied) 86 | ### 87 | ### 88 | ### 89 | ### Typical command lines (User tapper@tapper): 90 | ### 91 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/kernel/generic/test 92 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/kernel/generic/test -Dtests_host=hackbench 93 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/kernel/generic/test -Dmachines=lemure 94 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/kernel/generic/test -Dmachines=lemure -Dtests_host=hackbench 95 | ### 96 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/kernel/generic/test -Dkernelpkg=/data/tapper/live/repository/packages/kernel/sles11/i686/linux-3.3_rc3-94-g612b850.2012-02-10.i686.tgz 97 | # Builder details: 98 | # builder: [% builder %] 99 | # repository: [% repository %] 100 | # type: [% type %] 101 | 102 | [%- FOREACH distro = AllDistros %] 103 | [%- PROCESS 'include/distrodetails' -%] 104 | # distro: [% distro %] 105 | # install_file: [% install_file %] 106 | # install_repo: [% install_repo %] 107 | # install_opts: [% install_opts %] 108 | # kernel: [% kernel %] 109 | # initrd: [% initrd %] 110 | [%- END %] 111 | 112 | [%- FOREACH machine = AllMachines %] 113 | [%- FOREACH hostbootparam_entry = AllHostBootparams %] 114 | [%- FOREACH distro = AllDistros %] 115 | # machine: [% machine %] 116 | # distro: [% distro %] 117 | --- 118 | type: multitest 119 | description: 120 | shortname: "[% fullname %]" 121 | topic: "[% fullname %][% IF hostbootparam_entry %] [% hostbootparam_entry %][% END %]" 122 | queue: [% queue %] 123 | requested_hosts_all: 124 | - [% machine %] 125 | preconditions: 126 | - 127 | [%- IF distro == 'baseimage' %] 128 | precondition_type: image 129 | image: [% current_SLES11 %] 130 | partition: 131 | - testing 132 | - sda2 133 | - hda2 134 | mount: / 135 | arch: linux64 136 | [% ELSE %] 137 | grub_text: "timeout 2\n\ntitle [% distro %] Testing\nkernel [% kernel %] [% install_file %] [% install_repo %] [% install_opts %] nomodeset $HOSTOPTIONS $TAPPER_OPTIONS\ninitrd [% initrd %]\n" 138 | name: autoinstall-[% distro %] 139 | precondition_type: autoinstall 140 | timeout: 7200 141 | [% END %][%# baseimage %] 142 | [%# Currently we allow distro + additional kernelpkg ===> not sure whether that really works! %] 143 | [%- IF kernelpkg %] 144 | - 145 | precondition_type: package 146 | filename: [% kernelpkg %] 147 | [% ELSIF builder %] 148 | - 149 | precondition_type: produce 150 | producer: Builder 151 | type: [% type %] 152 | repository: [% repository %] 153 | buildserver: [% buildserver %] 154 | [%- IF builder_changeset %] 155 | version: [% builder_changeset %] 156 | [% END -%] 157 | [% END -%][%# builder %] 158 | [%- IF kernelpkg || builder %] 159 | - 160 | precondition_type: package 161 | filename: kernel/linux-firmware/firmware.tgz 162 | - 163 | precondition_type: copyfile 164 | name: /data/tapper/live/repository/packages/tapperutils/kernel/gen_initrd.sh 165 | dest: /bin/ 166 | protocol: local 167 | - 168 | precondition_type: exec 169 | filename: /bin/gen_initrd.sh 170 | - 171 | precondition_type: copyfile 172 | protocol: local 173 | name: /data/tapper/live/repository/testprograms/utilities/* 174 | dest: / 175 | [% END %][%# baseimage %] 176 | [%- FOREACH package0 = AllPackagesHost %] 177 | - 178 | precondition_type: package 179 | name: [% package0 %] 180 | [% END -%][%# FOREACH AllPackagesHost %] 181 | [%- FOREACH test0 = AllTestsHost %] 182 | [%- IF test0.match('^/') %][% dirname0 = test0.replace('/[^/]+$', '') %][% basename0 = test0.replace('.*/([^/]+)$', '$1') %] 183 | 184 | # HOST TEST PREPARE: [% test0 %] 185 | # (direct test script) 186 | - 187 | precondition_type: copyfile 188 | protocol: local 189 | name: [% dirname0 %]/* 190 | dest: / 191 | [% END -%][%# IF test0.match %] 192 | [% END -%][%# FOREACH AllTestsHost preconditions %] 193 | - 194 | precondition_type: testprogram 195 | program: /data/tapper/autoreport/metainfo.t 196 | chdir: AUTO 197 | [%- FOREACH test0 = AllTestsHost %] 198 | 199 | # HOST TEST: [% test0 %] 200 | [%- IF test0.match('^/') %][% dirname0 = test0.replace('/[^/]+$', '') %][% basename0 = test0.replace('.*/([^/]+)$', '$1') %] 201 | # (direct test script) 202 | - 203 | precondition_type: testprogram 204 | program: /[% basename0 %] 205 | timeout: [% test_timeout %] 206 | chdir: AUTO 207 | [%- IF tests_host_capture %] 208 | capture: [% tests_host_capture %] 209 | [% END %] 210 | [%- IF tests_host_params %] 211 | parameters: 212 | [%- FOREACH param0 = AllTestsParamsHost %] 213 | - [% param0 %] 214 | [% END %] 215 | [% END %] 216 | [% ELSE %] 217 | # (autotest) 218 | - 219 | precondition_type: testprogram 220 | program: /opt/tapper/bin/tapper-testsuite-autotest 221 | timeout: [% test_timeout %] 222 | parameters: 223 | - --source_url 224 | - file:///data/tapper/live/repository/packages/autotest/renormalist-autotest-039effb.tar.gz 225 | - --test 226 | - [% test0 %] 227 | [% END -%][%# IF test0.match %] 228 | [% END -%][%# FOREACH AllTestsHost testprograms %] 229 | - 230 | precondition_type: grub 231 | config: | 232 | timeout 3 233 | default 0 234 | title KERNEL [% kernel_version %] 235 | root $grubroot 236 | kernel /boot/vmlinuz root=$root $HOSTOPTIONS debug ignore_loglevel 237 | initrd /boot/initrd 238 | [%- END %][%# FOREACH AllDistros %] 239 | [%- END %][%# FOREACH AllHostBootparams %] 240 | [%- END %][%# FOREACH AllMachines %] 241 | -------------------------------------------------------------------------------- /examples/testplans/topic/xen/generic/guest-start-template.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | #################################################### 4 | # 5 | # Starter script for Xen 6 | # 7 | # - based on autoreport to report its own success 8 | # - assumes "xl" frontend 9 | # - on SLES 11 SP(1,2) distro-xen it uses xm 10 | # 11 | #################################################### 12 | 13 | . /virt/tapper-autoreport --import-utils 14 | 15 | xlxm=$(which xl) 16 | 17 | WE_ARE_ON_SLES11_AUTOINSTALL=0 18 | if grep -q "SUSE Linux Enterprise Server 11 SP" /etc/issue ; then 19 | if [ -e /root/postinstall.log ] ; then 20 | WE_ARE_ON_SLES11_AUTOINSTALL=1 21 | xlxm=$(which xm) 22 | fi 23 | fi 24 | 25 | SUITENAME="$(basename $xlxm)-create-__GUESTNR__-__GUESTNAME__" 26 | DONTREPEATMETA=1 27 | GUESTSTARTUPHOOKDIR=__GUESTSTARTUPHOOKDIR__ 28 | IOMMUV2=__IOMMUV2__ 29 | 30 | guestconfig="/virt/guest-__GUESTNR__.svm" 31 | log="/virt/$SUITENAME.log" 32 | echo "" > $log 33 | 34 | [ -x "$xlxm" ] 35 | require_ok $? "$(basename $xlxm) available" 36 | 37 | upload_dmesg="/virt/upload-xen-dmesg.sh" 38 | if [ -e $upload_dmesg ] ; then $upload_dmesg ; fi 39 | 40 | diag "SLES 11 autoinstall: $WE_ARE_ON_SLES11_AUTOINSTALL" 41 | 42 | if [ "x$WE_ARE_ON_SLES11_AUTOINSTALL" == "x0" ] ; then 43 | echo "/etc/init.d/xend stop" >> $log 44 | echo "------------------------------------------------------" >> $log 45 | echo "" >> $log 46 | /etc/init.d/xend stop >> $log 2>&1 47 | sleep 10 # grace period 48 | fi 49 | 50 | if [ -d "$GUESTSTARTUPHOOKDIR" ] ; then 51 | ( 52 | cd $GUESTSTARTUPHOOKDIR 53 | for startupscript in $(ls -1 | grep -v '^tapper-autoreport$' | sort); do 54 | ./$startupscript 55 | done 56 | ) 57 | fi 58 | 59 | if [[ ${IOMMUV2} == 1 ]]; then 60 | #enable passthrough mode for gfx 61 | echo "IOMMUV2: $IOMMUV2" >> $log 62 | echo "IOMMUV2: enable passthrough mode for gfx" >> $log 63 | bdf=$(lspci -n | grep "1002:6798" | head -1 | cut -d" " -f1) 64 | echo "IOMMUV2: $bdf" >> $log 65 | 66 | echo -n 0000:$bdf > /sys/bus/pci/drivers/pciback/new_slot 67 | echo -n 0000:$bdf > /sys/bus/pci/drivers/pciback/bind 68 | 69 | echo "pci = ['$bdf']" >> $guestconfig 70 | echo "guest_iommu = 1" >> $guestconfig 71 | fi 72 | 73 | # provide meta information to guest 74 | outputdir=$(/virt/autoreport-utils tapper_output_dir)/meta 75 | mkdir -p $outputdir 76 | _name=tapper_all_meta 77 | _meta="$outputdir/$_name.txt" 78 | _info="$outputdir/tapper_setup.txt" 79 | if [ ! -e "$_meta" ] ; then 80 | SUITENAME=$_name /virt/autoreport-utils $_name > $_meta 81 | fi 82 | if [ ! -e "$_info" ] ; then 83 | touch $_info 84 | echo "virtualization: xen" >> $_info 85 | echo "host: __DISTRO__" >> $_info 86 | echo "builder: __BUILDER__" >> $_info 87 | echo "builderchangeset: __BUILDERCHANGESET__" >> $_info 88 | echo "xenpkg: __XENPKG__" >> $_info 89 | echo "pvops: __PVOPS__" >> $_info 90 | echo "iommuv2: __IOMMUV2__" >> $_info 91 | echo "guestvcpus: __GUESTVCPUS__" >> $_info 92 | echo "guestmemory: __GUESTMEMORY__" >> $_info 93 | fi 94 | echo "guest-__GUESTNR__: __GUESTNAME__" >> $_info 95 | 96 | echo "" >> $log 97 | echo "" >> $log 98 | echo "*** $xlxm create $guestconfig ***" >> $log 99 | echo "------------------------------------------------------" >> $log 100 | echo "" >> $log 101 | $xlxm create $guestconfig >> $log 2>&1 102 | ok $? "start guest $guestconfig" 103 | 104 | . /virt/tapper-autoreport $guestconfig $log 105 | -------------------------------------------------------------------------------- /examples/testplans/topic/xen/generic/guest-template.svm: -------------------------------------------------------------------------------- 1 | builder = "hvm" 2 | vif = [ "mac=52:54:00:01:__HOSTNRHEX__:__GUESTNRHEX__,bridge=xenbr0" ] 3 | vnc = 1 4 | vnclisten = "0.0.0.0" 5 | vncpasswd = "" 6 | serial = "file:/tmp/guest__GUESTNR__.fifo" 7 | monitor = 1 8 | usb = 1 9 | usbdevice = "tablet" 10 | name = "guest-__GUESTNR__-__GUESTNAME__" 11 | disk = [ "__DISKTEMPLATE1__", 12 | "__DISKTEMPLATE2__" ] 13 | boot = "c" 14 | acpi = 1 15 | apic = 1 16 | pae = 1 17 | hpet = 0 18 | shadow_memory = 10 19 | memory = __GUESTMEMORY__ 20 | vcpus = __GUESTVCPUS__ 21 | hap = 1 22 | -------------------------------------------------------------------------------- /examples/testplans/topic/xen/generic/test: -------------------------------------------------------------------------------- 1 | [%#- -*- mode: tt -*- %] 2 | [%- PROCESS 'include/defaults' -%] 3 | [% USE Autoformat %] 4 | [%- IF machine != '' %][% THROW LEGACY "Use '-Dmachines=...' (plural) now.\n" %][% END -%] 5 | [%- IF tests_dom0 != '' %][% THROW LEGACY "Use '-Dtest_host=...' now.\n" %][% END -%] 6 | [%- IF tests_domu != '' %][% THROW LEGACY "Use '-Dtests_guest=...' now.\n" %][% END -%] 7 | [%- IF Timeout != '' %][% THROW LEGACY "Use '-Dtest_timout=...' now.\n" %][% END -%] 8 | [%- IF tests != '' %][% THROW AMBIGUITY "Use '-Dtests_host=...' or '-Dtests_guest=...'.\n" %][% END -%] 9 | [%- IF host != '' %][% THROW AMBIGUITY "Use '-Dmachines=...'.\n" %][% END -%] 10 | [%- IF hosts != '' %][% THROW AMBIGUITY "Use '-Dmachines=...'.\n" %][% END -%] 11 | [%- IF queues != '' %][% THROW AMBIGUITY "Use '-Dqueue=name' (singular).\n" %][% END -%] 12 | [%- IF queue.match(',') %][% THROW AMBIGUITY "Use '-Dqueue=name' (only one queue).\n" %][% END -%] 13 | [%- IF queue == '' %][% queue = 'AdHoc' %][% END -%] 14 | [%- IF machines == '' %][% machines = 'limete' %][% END -%] 15 | [%- IF distros == '' %][% distros = 'baseimage' %][% END -%] 16 | [%- IF buildserver == '' %][% buildserver = 'acid' %][% END -%][%# acid=64bit, selen=32bit %] 17 | [%- IF guestimages == '' %][% guestimages = 'redhat_rhel6u2_64b_raw.img,suse_sles11_sp2_64b_raw.img,redhat_rhel6u2_32b_raw.img' %][% END -%] 18 | [%- IF guestmemory == '' %][% guestmemory = '1024' %][% END -%] 19 | [%- IF guestvcpus == '' %][% guestvcpus = '2' %][% END -%] 20 | [%- IF gueststarthookdir == '' %][% gueststarthookdir = '' %][% END -%] 21 | [%- IF packages_host == '' %][% packages_host = '' %][% END -%] 22 | [%- IF packages_guest == '' %][% packages_guest = '' %][% END -%] 23 | [%- IF use_compressed_images == '' %][% use_compressed_images = '0' %][% END -%] 24 | [%- IF tests_host == '' %][% tests_host = '' %][% END -%][%# 'hackbench,unixbench,dbench,tiobench,iozone,reaim,aiostress,compilebench' %] 25 | [%- IF tests_guest == '' %][% tests_guest = '' %][% END -%][%# 'hackbench,unixbench,dbench,tiobench,iozone,reaim,aiostress,compilebench' %] 26 | [%- IF test_timeout == '' %][% IF tests_guest.match('winsst') %][% test_timeout = '36000' %][% ELSE %][% test_timeout = '10800' %][% END -%][% END -%] 27 | [%- shorttitle_test_host = BLOCK %][% IF tests_host.match(',') %]MULTI[% ELSE %][% tests_host.replace('.*/([^/]+)$', '$1').replace('^(xen|kvm)-','').replace('\.sh$','').replace('\.t$','').replace('[-_]','') %][% END %][% END -%] 28 | [%- shorttitle_test_guest = BLOCK %][% IF tests_guest.match(',') %]MULTI[% ELSE %][% tests_guest.replace('.*/([^/]+)$', '$1').replace('^(xen|kvm)-','').replace('\.sh$','').replace('\.t$','').replace('[-_]','') %][% END %][% END -%] 29 | [%- shorttitle_benchmark_guest = BLOCK %][% IF ourbenchmarks_guest %][% IF ourbenchmarks_guest.match(',') %]MULTI[% ELSE %][% ourbenchmarks_guest.replace('.*/([^/]+)$', '$1').replace('^(xen|kvm)-','').replace('\.sh$','').replace('\.t$','').replace('[-_]','') %][% END %][% END -%][% END -%] 30 | [%- IF title == '' %][% title = BLOCK %]Xen[%- IF shorttitle_test_host != '' %]-[% shorttitle_test_host %][% ELSE %][% END %][%- IF shorttitle_test_guest != '' %]-[% shorttitle_test_guest %][% END %][% END %][% END -%] 31 | [%- IF titlesuffix == '' %][% titlesuffix = BLOCK %][%- IF shorttitle_benchmark_guest != '' %][% shorttitle_benchmark_guest %][% END %][% END %][% END -%] 32 | [%- IF builder == '' && distros == 'baseimage' && xenpkg == '' %][% builder = '4upv' %][% END -%] 33 | [%- IF distros == 'baseimage' %][% disktemplate1 = '/virt/guest-__GUESTNR__.img,raw,hda,w' %][% disktemplate2 = '/virt/guest-__GUESTNR__-auxiliary.img,raw,hdb,w' %][% ELSE %][% disktemplate1 = 'file:/virt/guest-__GUESTNR__.img,hda,w' %][% disktemplate2 = 'file:/virt/guest-__GUESTNR__-auxiliary.img,hdb,w' %][% END -%][%# "file:" is the deprecated thing, but still needed in distros -%] 34 | [% repository = builder %] 35 | [% type = 'xen' %] 36 | [%- AllMachines = machines.split(',') || [] -%] 37 | [%- AllDistros = distros.split(',') || [] -%] 38 | [%- AllPackagesHost = packages_host.split(',') || [] -%] 39 | [%- AllPackagesGuest = packages_guest.split(',') || [] -%] 40 | [%- AllTestsHost = tests_host.split(',') || [] -%] 41 | [%- AllTestsGuest = tests_guest.split(',') || [] -%] 42 | [%- GuestImageNames = guestimages.split(',') || [] -%] 43 | [%- AllHostBootparams = hostbootparams.split(';') || [] -%][%# SIC, semicolon! We need ',' inside bootparams. -%] 44 | [%- AllOURBenchmarksGuest = ourbenchmarks_guest.split(',') || [] -%] 45 | [%- AllOURBenchmarksGuestParams = ourbenchmarksparams_guest.split(',') || [] -%] 46 | [%- GuestImageNames.unshift('NO_IMAGE_AT_INDEX_0') %][%# interesting index starts at 1, for readability -%] 47 | [% fullname = BLOCK %][% title %][% IF titlesuffix %]-[% titlesuffix %][% END %][% END %] 48 | [% baseimage = current_SLES11 %] 49 | [%- IF AllHostBootparams.size == 0 %][%- AllHostBootparams.unshift('') %][% END -%][%# to not empty the loop over it -%] 50 | ### 51 | ### Run a Xen environment with tests. 52 | ### [topic.xen.generic.test] 53 | ### 54 | ### 55 | ### Name : [% fullname %] 56 | ### Host : [% current_SLES11 %] 57 | ### Guests : [% guestimages %] 58 | ### 59 | ### Optional params: 60 | ### 61 | ### -Dxenpkg= Filename of already existing Xen package; default: build one 62 | ### -Dbuilder= Key what type of Xen to build; default: 4upv 63 | ### -Dbuilder_changeset= Changeset of Xen to build; default: 64 | ### 65 | ### -Dhostbootparams= Kernel boot parameter to host, SEMICOLON[sic] separated; default: 66 | ### -Dtest_timeout= Timeout for tests; default: 10800 (3h) 67 | ### 68 | ### -Dtests_host= Workload names, comma separated, see list below; default: 69 | ### -Dtests_host_capture=tap Capture and send TAP output of host tests; default: 70 | ### 71 | ### -Dtests_guest= Workload names, comma separated, see list below; default: 72 | ### -Dtests_guest_capture=tap Capture and send TAP output of guest tests; default: 73 | ### 74 | ### -Ddistros= Host distro names, comma separated; see list below; default: sles_11.2_64 75 | ### -Dguestimages= Images for guests, comma separated; see list below; default: sles_11.2_64 76 | ### -Dguestmemory= Memory in Megabyte assigned to each guest; default: 1024 77 | ### -Dguestvcpus= Number of virtual CPUs assigned to each guest; default: 2 78 | 79 | ### -Dmachines= Machine names, comma separated; default: limete 80 | ### -Dqueue= Queue to use; default: AdHoc 81 | ### -Dtitlesuffix= Suffix to be used in testplan and testrun names; default: 82 | ### 83 | ### Available values: 84 | ### 85 | ### builder: 86 | [%- FILTER autoformat(all => 1) %] 87 | ### [% FOREACH m = builder_list -%][% m %], [% END %] 88 | [% END %] 89 | ### Mnemonics: 90 | ### - numbers mean the xen release 4.0, 4.1, etc. 91 | ### - "pv" means "pvops" 92 | ### - "u" means "unstable" 93 | ### - "s" means "staging" 94 | ### - "osvw" means "the osvw branch" (implicit "pv") 95 | ### - "vmcb" means "with vmcb cleanbit debug patch" (implicit "pv") 96 | ### - "decode" means "with decode assist debug patch" (implicit "pv") 97 | ### machines: 98 | [%- FILTER autoformat(all => 1) %] 99 | ### [% FOREACH m = machine_list -%][% m %], [% END %] 100 | [% END %] 101 | ## distros: 102 | [%- FILTER autoformat(all => 1) %] 103 | ## [% FOREACH d = distro_list -%][% d %], [% END %] 104 | [% END %] 105 | ## guestimages: 106 | [%- FILTER autoformat(all => 1) %] 107 | ## [% FOREACH d = image_list -%][% d %], [% END %] 108 | [% END %] 109 | ### tests (tests_host, tests_guest): 110 | ### - non-path words are interpreted as Autotest subtests: 111 | ### 112 | [%- FILTER autoformat(all => 1) %] 113 | ### [% FOREACH t = useful_autotest_tests -%][% t %], [% END %] 114 | [%- END -%] 115 | ### 116 | ### - absolute path names, like "/data/tapper/autoreport/mytest.t" 117 | ### are interpreted as script to execute (and every file inside 118 | ### that path/* will also be copied) 119 | ### 120 | ### 121 | ### 122 | ### Typical command lines (User tapper@tapper): 123 | ### 124 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/xen/generic/test 125 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/xen/generic/test -Dtests_guest=hackbench 126 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/xen/generic/test -Dmachines=lemure 127 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/xen/generic/test -Dmachines=lemure -Dtests_host=hackbench 128 | ### 129 | ### $ tapper-testrun newtestplan --file /data/tapper/live/testplan/topic/xen/generic/test -Dxenpkg=/data/tapper/live/repository/packages/xen/sles11/x86_64/xen-4.0-testing-pvops/xen-4.0-testing-pvops.2012-01-18.21560_271e30252c16.x86_64.tgz 130 | [% IF xenpkg.match('pvops') or builder.match('pv') or builder.match('osvw') or builder.match('vmcb') or builder.match('decode') %] 131 | [% pvops = '1' %] 132 | [% pvopskernelsuffix = '-pvops' %] 133 | [% ELSE %] 134 | [% pvops = '0' %] 135 | [% pvopskernelsuffix = '' %] 136 | [% END %] 137 | [% IF xenpkg.match('iommuv2') or tests_guest.match('iommuv2') or guestimages.match('iommuv2') %] 138 | [% iommuv2 = '1' %] 139 | [% ELSE %] 140 | [% iommuv2 = '0' %] 141 | [% END %] 142 | 143 | # Builder details: 144 | # buildserver: [% buildserver %] 145 | # builder: [% builder %] 146 | # repository: [% repository %] 147 | # type: [% type %] 148 | # pvops: [% pvops %] 149 | [%- FOREACH test0 = AllTestsHost %] 150 | # test0: [% test0 %] 151 | [%- END %] 152 | 153 | [%- FOREACH testu = AllTestsGuest %] 154 | # testu: [% testu %] 155 | [%- END %] 156 | 157 | [%- FOREACH benchmarku = AllOURBenchmarksGuest %] 158 | # benchmarku: [% benchmarku %] 159 | [%- FOREACH benchmarkparamu = AllOURBenchmarksGuestParams %] 160 | # [% benchmarkparamu %] 161 | [%- END %] 162 | [%- END %] 163 | 164 | [%- FOREACH distro = AllDistros %] 165 | [%- PROCESS 'include/distrodetails' -%] 166 | # distro: [% distro %] 167 | # install_file: [% install_file %] 168 | # install_repo: [% install_repo %] 169 | # install_opts: [% install_opts %] 170 | # kernel: [% kernel %] 171 | # initrd: [% initrd %] 172 | [%- END %] 173 | 174 | [%- FOREACH machine = AllMachines %] 175 | [%- FOREACH hostbootparam_entry = AllHostBootparams %] 176 | [%- FOREACH distro = AllDistros %] 177 | [% distroname = distro %] 178 | [% IF distroname == 'baseimage' %][% distroname = baseimage.replace('.*/([^/]+)$', '$1').replace('.tar.gz','') %][% END %] 179 | [% machine_id = BLOCK %][% testrundb_hostnames.$machine.id | format("%02d") %][% END %] 180 | [% machine_id_hex = BLOCK %][% testrundb_hostnames.$machine.id | format("%02x") %][% END %] 181 | # machine: [% machine %] (ID: [% machine_id %]/0x[% machine_id_hex %]) 182 | # distro: [% distro %] 183 | # distroname: [% distroname %] 184 | --- 185 | type: multitest 186 | description: 187 | shortname: "[% fullname %]" 188 | topic: "[% fullname %][% IF hostbootparam_entry %] [% hostbootparam_entry %][% END %]" 189 | queue: [% queue %] 190 | requested_hosts_all: 191 | - [% machine %] 192 | preconditions: 193 | - 194 | precondition_type: virt 195 | name: "Xen-hosttest:[% shorttitle_test_host %];guesttest:[% shorttitle_test_guest %][% IF shorttitle_benchmark_guest %];guestbenchmark:[% shorttitle_benchmark_guest %][% END %]" 196 | host: 197 | root: 198 | [%- IF distro == 'baseimage' %] 199 | precondition_type: image 200 | image: [% baseimage %] 201 | partition: /dev/sda2 202 | mount: / 203 | arch: linux64 204 | [% ELSE %] 205 | grub_text: "timeout 2\n\ntitle [% distro %] Testing\nkernel [% kernel %] [% install_file %] [% install_repo %] [% install_opts %] nomodeset $HOSTOPTIONS $TAPPER_OPTIONS\ninitrd [% initrd %]\n" 206 | name: autoinstall-[% distro %] 207 | precondition_type: autoinstall 208 | timeout: 7200 209 | [% END %] 210 | preconditions: 211 | [%- IF xenpkg %] 212 | - 213 | precondition_type: package 214 | filename: [% xenpkg %] 215 | [% ELSIF builder %] 216 | - 217 | precondition_type: produce 218 | producer: Builder 219 | type: [% type %] 220 | repository: [% repository %] 221 | buildserver: [% buildserver %] 222 | [%- IF builder_changeset %] 223 | version: [% builder_changeset %] 224 | [% END -%] 225 | [% END -%][%# xenpkg %] 226 | [%- IF xenpkg || builder %] 227 | - 228 | precondition_type: package 229 | filename: tapperutils/sles10/xen_installer_suse.tar.gz 230 | - 231 | precondition_type: exec 232 | filename: /bin/xen_install.sh 233 | [%- IF iommuv2 %] 234 | options: 235 | - xen-iommuv2-pvops 236 | [%- ELSIF pvops %] 237 | options: 238 | - pvops 239 | [% ELSE %] 240 | options: ~ 241 | [% END -%][%# iommuv2/pvops %] 242 | [% END -%][%# xenpkg %] 243 | - 244 | precondition_type: copyfile 245 | protocol: local 246 | name: /data/tapper/live/repository/testprograms/utilities/* 247 | dest: / 248 | [%- FOREACH guest IN [ 1 .. GuestImageNames.max ] %] 249 | [% guest = BLOCK %][% guest | format('%02d') %][% END %] 250 | [% guestnrhex = BLOCK %][% guest | format('%02x') %][% END %] 251 | [% guest_image_basename = GuestImageNames.$guest.replace('^(.*)\.[^.]+$', '$1') %] 252 | # guest [% guest %] config preparation 253 | 254 | [%- IF use_compressed_images %] 255 | - 256 | precondition_type: exec 257 | filename: /bin/mv 258 | options: 259 | - /virt/[% guest_image_basename %].img 260 | - /virt/[% guest %].img 261 | [% END %] 262 | - 263 | precondition_type: copyfile 264 | name: /data/tapper/autoreport/tapper-autoreport 265 | dest: /virt/ 266 | protocol: local 267 | - 268 | precondition_type: copyfile 269 | name: /data/tapper/autoreport/autoreport-utils 270 | dest: /virt/ 271 | protocol: local 272 | - 273 | precondition_type: copyfile 274 | name: /data/tapper/live/testplan/topic/xen/generic/upload-xen-dmesg.sh 275 | dest: /virt/ 276 | protocol: local 277 | - 278 | precondition_type: copyfile 279 | name: /data/tapper/live/testplan/topic/xen/generic/guest-start-template.sh 280 | dest: /virt/guest-start-[% guest %].sh 281 | protocol: local 282 | - 283 | precondition_type: copyfile 284 | name: /data/tapper/live/testplan/topic/xen/generic/guest-template.svm 285 | dest: /virt/guest-[% guest %].svm 286 | protocol: local 287 | - 288 | precondition_type: exec 289 | filename: /usr/bin/perl 290 | options: 291 | - -pni 292 | - -e 293 | [%-# Order matters! Substitute __DISKTEMPLATEx__ first! %] 294 | - "'s{__DISKTEMPLATE1__}{[% disktemplate1 %]}g; s{__DISKTEMPLATE2__}{[% disktemplate2 %]}g; s/__IOMMUV2__/[% iommuv2 %]/g; s/__HOSTNRHEX__/[% machine_id_hex %]/g; s/__DISTRO__/[% distroname %]/g; s/__PVOPS__/[% pvops %]/g; s/__BUILDER__/[% builder %]/g; s/__BUILDERCHANGESET__/[% builder_changeset %]/g; s{__XENPKG__}{[% xenpkg %]}g; s/__GUESTNRHEX__/[% guestnrhex %]/g; s/__GUESTNR__/[% guest %]/g; s/__GUESTNAME__/[% GuestImageNames.$guest %]/g; s/__GUESTMEMORY__/[% guestmemory %]/g; s/__GUESTVCPUS__/[% guestvcpus %]/g; s{__GUESTSTARTHOOKDIR__}{[% gueststarthookdir %]}g'" 295 | - /virt/guest-start-[% guest %].sh 296 | - /virt/guest-[% guest %].svm 297 | [%- END -%][%# FOREACH guest %] 298 | [%- FOREACH package0 = AllPackagesHost %] 299 | - 300 | precondition_type: package 301 | name: [% package0 %] 302 | [% END -%][%# FOREACH AllPackagesHost %] 303 | [%- FOREACH test0 = AllTestsHost %] 304 | [%- IF ( test0.match('^/') && ! test0.match('^/opt/tapper') ) %][% dirname0 = test0.replace('/[^/]+$', '') %][% basename0 = test0.replace('.*/([^/]+)$', '$1') %] 305 | # HOST TEST PREPARE: [% test0 %] 306 | # (direct test script) 307 | - 308 | precondition_type: copyfile 309 | protocol: local 310 | name: [% dirname0 %]/* 311 | dest: / 312 | [% END -%][%# IF test0.match %] 313 | [% END -%][%# FOREACH AllTestsHost preconditions %] 314 | testprogram_list: 315 | - 316 | program: /data/tapper/autoreport/metainfo.t 317 | chdir: AUTO 318 | [%- FOREACH test0 = AllTestsHost %] 319 | # HOST TEST: [% test0 %] 320 | [%- IF test0.match('^/') %][% dirname0 = test0.replace('/[^/]+$', '') %][% basename0 = test0.replace('.*/([^/]+)$', '$1') %] 321 | # (direct test script) 322 | - 323 | program: /[% basename0 %] 324 | [%- IF tests_host_capture %] 325 | capture: [% tests_host_capture %] 326 | [% END %] 327 | [% ELSE %] 328 | # (autotest) 329 | - 330 | program: /opt/tapper/bin/tapper-testsuite-autotest 331 | environment: 332 | http_proxy: http://gwo:3128 333 | https_proxy: http://gwo:3128 334 | parameters: 335 | - --source_url 336 | - file:///data/tapper/live/repository/packages/autotest/renormalist-autotest-039effb.tar.gz 337 | - --test 338 | - [% test0 %] 339 | [% END -%][%# IF test0.match %] 340 | timeout: [% test_timeout %] 341 | [% END -%][%# FOREACH AllTestsHost testprograms %] 342 | [%- IF GuestImageNames.max > 0 %] 343 | guests: 344 | [%- FOREACH guest IN [ 1 .. GuestImageNames.max ] %] 345 | [% guest = BLOCK %][% guest | format('%02d') %][% END %] 346 | [% guest_image_basename = GuestImageNames.$guest.replace('^(.*)\.[^.]+$', '$1') %] 347 | [% guest_image_compressed = BLOCK %][% guest_image_basename %].tgz[% END %] 348 | # GUEST [% guest %] preconditions 349 | - 350 | root: 351 | [%- IF use_compressed_images %] 352 | precondition_type: package 353 | url: nfs://osko:/export/image_files/official_testing/tared_testimages/[% guest_image_compressed %] 354 | target_directory: /virt 355 | [%- ELSE %] 356 | precondition_type: copyfile 357 | name: "osko:/export/image_files/official_testing/[% GuestImageNames.$guest %]" 358 | dest: /virt/guest-[% guest %].img 359 | protocol: nfs 360 | [%- END %] 361 | [%- IF GuestImageNames.$guest.match('_32b') %] 362 | arch: linux32 363 | [%- ELSIF GuestImageNames.$guest.match('windows') %] 364 | arch: windows 365 | [%- ELSE %] 366 | arch: linux64 367 | [%- END %] 368 | mountfile: /virt/guest-[% guest %]-auxiliary.img 369 | mounttype: raw 370 | config: 371 | exec: /virt/guest-start-[% guest %].sh 372 | [%- IF AllTestsGuest.size %][%# || AllOURBenchmarksGuest.size %] 373 | preconditions: 374 | [%- END %] 375 | [%- FOREACH packageU = AllPackagesGuest %] 376 | - 377 | precondition_type: package 378 | name: [% packageU %] 379 | [% END -%][%# FOREACH AllPackagesGuest %] 380 | [%- FOREACH testU = AllTestsGuest %] 381 | [%- IF ( testU.match('^/') && ! testU.match('^/opt/tapper') ) %][% dirnameU = testU.replace('/[^/]+$', '') %][% basenameU = testU.replace('.*/([^/]+)$', '$1') %] 382 | # GUEST TEST PREPARE: [% testU %] 383 | # (direct test script) 384 | - 385 | precondition_type: copyfile 386 | protocol: local 387 | name: [% dirnameU %]/* 388 | dest: / 389 | [%- END %][%# IF testU.match %] 390 | [%- END %][%# FOREACH AllTestsGuest preconditions %] 391 | testprogram_list: 392 | - 393 | program: /data/tapper/autoreport/metainfo.t 394 | chdir: AUTO 395 | [%- FOREACH testU = AllTestsGuest %] 396 | # GUEST TEST: [% testU %] 397 | [%- IF ( testU.match('^/') && ! testU.match('^/opt/tapper') ) %][% dirnameU = testU.replace('/[^/]+$', '') %][% basenameU = testU.replace('.*/([^/]+)$', '$1') %] 398 | # (direct test script) 399 | - 400 | program: /mnt/tapper/[% basenameU %] 401 | chdir: AUTO 402 | [%- IF tests_guest_capture %] 403 | capture: [% tests_guest_capture %] 404 | [% END %] 405 | [% ELSIF testU.match('^/opt/tapper') %] 406 | # (clientpkg test script) 407 | - 408 | program: [% testU %] 409 | [% ELSIF testU.match('winsst') %] 410 | # (winsst) 411 | - 412 | program: winsst 413 | runtime: 16000 414 | [% ELSE %] 415 | # (autotest) 416 | - 417 | program: /opt/tapper/bin/tapper-testsuite-autotest 418 | environment: 419 | http_proxy: http://gwo:3128 420 | https_proxy: http://gwo:3128 421 | parameters: 422 | - --source_url 423 | - [% autotest_snapshot_url %] 424 | - --test 425 | - [% testU %] 426 | [%- END %][%# IF testU.match %] 427 | timeout: [% test_timeout %] 428 | [%- END %][%# FOREACH AllTestsGuest testprogram s%] 429 | [%- FOREACH benchmarkU = AllOURBenchmarksGuest %] 430 | [%- ourbenchmark = benchmarkU -%][%# 'include/defaultbenchmarks' uses ourbenchmark %] 431 | [%- PROCESS 'include/defaultbenchmarks' -%] 432 | # GUEST BENCHMARK: [% ourbenchmark %] 433 | - 434 | program: [% benchmarkscript %] 435 | timeout: [% test_timeout %] 436 | [%- IF AllOURBenchmarksGuestParams.size %] 437 | parameters: 438 | [%- END %] 439 | [%- FOREACH benchmarkparamU = AllOURBenchmarksGuestParams %] 440 | - [% benchmarkparamU %] 441 | [%- END %][%# FOREACH AllOURBenchmarksGuestParams %] 442 | [%- END %][%# FOREACH AllOURBenchmarksGuest benchmarkU %] 443 | [%- END %][%# FOREACH guest %] 444 | [%- END %][%# IF GuestImageNames.max > 0 %] 445 | [%- IF distro == 'baseimage' %] 446 | - 447 | precondition_type: grub 448 | shortname: grub 449 | config: "timeout 3\ndefault 0\ntitle XEN\n root $grubroot\n kernel /boot/xen.gz $XENHOSTOPTIONS console=com1 com1=115200 [% hostbootparam_entry %] loglvl=all guest_loglvl=all noreboot\n module /boot/vmlinuz[% pvopskernelsuffix %] $TAPPER_OPTIONS root=$root console=hvc0 nomodeset\n module /boot/initrd[% pvopskernelsuffix %]\n" 450 | # reboot=force 451 | [%- END %] 452 | [%- END %][%# FOREACH AllDistros %] 453 | [%- END %][%# FOREACH AllHostBootparams %] 454 | [%- END %][%# FOREACH AllMachines %] 455 | -------------------------------------------------------------------------------- /examples/testplans/topic/xen/generic/upload-xen-dmesg.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | #################################################### 4 | # Upload Xen dmesg 5 | #################################################### 6 | 7 | . /virt/tapper-autoreport --import-utils 8 | 9 | DONTREPEATMETA=1 10 | 11 | xl=$(which xl) 12 | require_ok $? "xl available" 13 | 14 | # upload dmesg shortly after start, but only once 15 | xendmesglog="/virt/xen-dmesg.log" 16 | if [ ! -e $xendmesglog ] ; then 17 | $xl dmesg > $xendmesglog 18 | . /virt/tapper-autoreport $xendmesglog 19 | fi 20 | 21 | # silent exit, we are not a PRC visible testscript anyway 22 | -------------------------------------------------------------------------------- /lib/Tapper.pm: -------------------------------------------------------------------------------- 1 | use strict; 2 | use warnings; 3 | package Tapper; 4 | # ABSTRACT: A flexible and open test infrastructure 5 | 6 | 1; 7 | 8 | =head1 ABOUT 9 | 10 | Tapper is a modular, flexible and open test infrastructure. 11 | 12 | Its only primary assumption is the ubiquitous use of the B (TAP). Internally it is based on technology known 14 | from the CPAN testing infrastructure, extending it with automation and 15 | advanced querying. 16 | 17 | It allows to setup a test infrastructure, consisting of 18 | 19 | =over 4 20 | 21 | =item a central TAP database 22 | 23 | to where tests can send their results for storage and later 24 | evaluation, using a L based query language over a 25 | history of L. 26 | 27 | =item extremely lightweight APIs 28 | 29 | to allow QA in very low-level, toolchain constrained environments, 30 | like OS testing or embedded platforms 31 | 32 | =item support for "classic" userland, OS, and virtualization testing 33 | 34 | =item an optional advanced automation layer 35 | 36 | =back 37 | 38 | Tapper originated in the AMD Operating System Research Center (OSRC), 39 | where it is used for extreme continuous integration testing of 40 | everything: from software toolchains over Linux to Xen and KVM. 41 | 42 | It is also used in the Perl benchmarking infrastructure 43 | L. 44 | 45 | Most of Tapper is OS and platform independent (x86, ARM) to cover 46 | broad range of test targets. Only the central automation layer (MCP) 47 | is a bit Linux centric in its use of external dependencies. 48 | 49 | =head1 MORE 50 | 51 | =over 4 52 | 53 | =item L 54 | 55 | =item L 56 | 57 | =item L 58 | 59 | =back 60 | 61 | =head1 GENERAL TAPPER SUPPORT 62 | 63 | =head2 IRC 64 | 65 | Via IRC you can contact us in channel 66 | 67 | #tapper 68 | 69 | on 70 | 71 | irc.freenode.net 72 | 73 | although we are a bit busy these days (yes, working on&with Tapper) 74 | and suggest to first contact us per email. 75 | 76 | =head2 email 77 | 78 | tapper@tapper-testing.org 79 | 80 | =cut 81 | --------------------------------------------------------------------------------