├── README.md ├── httpdconf.sh ├── index.cgi ├── rollcopy.sh └── unpack-guides.sh /README.md: -------------------------------------------------------------------------------- 1 | __Scripts to copy [Rocks Clusters](http://www.rocksclusters.org) roll isos to a generic (linux) webserver for faster local hosting__ 2 | 3 | Purpose: Rocks 7.0 frontends require a web server to have rolls "installed" to download. Sometimes distance or 4 | other issues indicate that a local copy on an existing (non-Rocks) web-server would be much better. The scripts in this 5 | repository help with that. 6 | 7 | There are four scripts defined in this roll 8 | * rollcopy.sh - copy a roll iso contents for web serving 9 | * index.cgi - cgi to present roll directory in a standard format 10 | * httpdconf.sh (optional) - defines a virtual server httpd configuration file 11 | * unpack-guides (optional) - unpacks roll users guides into a web-servable directory 12 | 13 | __0. Preparation__ 14 | * clone this repository `git clone https://github.com/rocksclusters/roll-server.git` 15 | * download the roll isos from UCSD. 16 | 17 | __I. Copy Roll(s) to a web directory__ 18 | For this example, will assume the `/var/www/html/rocks/7.0` is the (already created) parent directory into 19 | which you want to copy rolls. The `rollcopy.sh` script will create the subdirectory `install/rolls`. 20 | to copy the iso file, `CentOS-7.4.1708-0.x86_64.disk1.iso` to the the release directory: 21 | ``` 22 | # rollcopy.sh CentOS-7.4.1708-0.x86_64.disk1.iso /var/www/html/rocks/7.0 23 | ``` 24 | Repeat the above for each roll iso that you want to copy 25 | 26 | __II. Make certain that the supplied `index.cgi` script gives the directory listing__ 27 | ``` 28 | # cp index.cgi /var/www/html/rocks/7.0/install/rolls 29 | ``` 30 | 31 | You will probably need to add something like the following in your Apache 32 | configuration, so that the directory index is given via the index.cgi 33 | 34 | ``` 35 | # allow all access to the rolls RPMS 36 | 37 | AddHandler cgi-script .cgi 38 | Options FollowSymLinks Indexes ExecCGI 39 | DirectoryIndex /rocks/7.0/install/rolls/index.cgi 40 | Allow from all 41 | 42 | ``` 43 | (See below for an example of how to test and verify that your index.cgi is being 44 | called properly) 45 | 46 | __III. (Optional) provide a virtual httpd server__ 47 | The script `httpdconf.sh` will write to standard output a reasonable stanza for Apache web server configuration. 48 | Suppose that you wanted the virtual server to be called `rocks-7.my.org` and have it used the directory above. 49 | Then 50 | ``` 51 | httpdconf.sh rocks-7-0.my.org /var/www/html/rocks/7.0 52 | ``` 53 | will generate an httpds conf file. It is your responsibility to place this on the appropriate directory and restart 54 | your webserver. 55 | 56 | __IV. (Optional) unpack the userguides__ 57 | You can unpack userguides 58 | ``` 59 | unpack-guides.sh /var/www/html/rocks/7.0/install/rolls /var/www/html/rocks/7.0 60 | ``` 61 | 62 | __V. Verifying that your output is correct (that is, index.cgi is being called)__ 63 | try 64 | ``` 65 | wget -O - http://central-7-0-x86-64.rocksclusters.org/install/rolls 66 | ``` 67 | and compare the output of the rocks central server listing of rolls to 68 | your website, something like 69 | ``` 70 | wget -O - http://rocks-7-0.my.org/install/rolls 71 | 72 | ``` 73 | You should see output very similar to 74 | ``` 75 | TTP request sent, awaiting response... 200 OK 76 | Length: 877 [text/html] 77 | Saving to: ‘STDOUT’ 78 | 79 | 0% [ ] 0 --.-K/s 82 | 85 | 88 | 91 | 94 | 97 | 100 | 103 | 106 | 109 | 112 | 115 | 118 | 121 | 124 | 127 | 100%[======================================>] 877 --.-K/s in 0s 128 | ``` 129 | __VI. Verifying that rolls directories can be listed__ 130 | 131 | The rolls themselves need be listed. Try the following 132 | ``` 133 | wget -O - http://central-7-0-x86-64.rocksclusters.org/install/rolls/base/7.0/x86_64 134 | ``` 135 | and verify that when you use your roll server instead that you get the same output 136 | 137 | __VII. Other items to check__ 138 | 139 | The following may need to be checked/changed for your setup: 140 | 141 | * Firewall needs modifying on your web server to allow http access (platform sepcific) 142 | 143 | * SELinux may need to be modified or turned off (e.g., ```setenforce Permissive```) 144 | 145 | 146 | 147 | -------------------------------------------------------------------------------- /httpdconf.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | VSERVER=$1 3 | DISTPATH=$2 4 | DOCROOT=$(readlink -f $2) 5 | cat << EOF | sed -e "s#@VSERVER@#$1#g" -e "s#@DOCROOT@#$DOCROOT#g" 6 | ### 7 | ## Definitions for @VSERVER@ 8 | ### 9 | 10 | ServerName @VSERVER@ 11 | DocumentRoot @DOCROOT@ 12 | 13 | 14 | 15 | Options FollowSymLinks Indexes ExecCGI 16 | AllowOverride None 17 | Order allow,deny 18 | Allow from all 19 | 20 | 21 | 22 | AllowOverride None 23 | SSLRequireSSL 24 | SSLVerifyClient None 25 | Allow from all 26 | 27 | # allow all access to the rolls RPMS 28 | 29 | DirectoryIndex /install/rolls/index.cgi 30 | Allow from all 31 | 32 | 33 | EOF 34 | -------------------------------------------------------------------------------- /index.cgi: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import os 4 | 5 | try: 6 | dir = os.environ['DOCUMENT_ROOT'] + os.environ['REQUEST_URI'] 7 | except: 8 | dir = '.' 9 | pass 10 | 11 | out = '' 12 | 13 | out += '' 14 | out += '' 15 | out += '
80 | area51/ 81 |
83 | base/ 84 |
86 | CentOS/ 87 |
89 | core/ 90 |
92 | fingerprint/ 93 |
95 | ganglia/ 96 |
98 | hpc/ 99 |
101 | htcondor/ 102 |
104 | kernel/ 105 |
107 | kvm/ 108 |
110 | openvswitch/ 111 |
113 | perl/ 114 |
116 | python/ 117 |
119 | sge/ 120 |
122 | Updates-CentOS-7.4.1708/ 123 |
125 | zfs-linux/ 126 |
' 16 | 17 | listing = os.listdir(dir) 18 | listing.sort(key=str.lower) 19 | for file in listing: 20 | if file not in [ 'index.cgi' ]: 21 | out += '' 29 | out += '\n' 30 | 31 | out += '
\n' 22 | 23 | if os.path.isdir(os.path.join(dir, file)): 24 | out += '%s/\n' % (file, file) 25 | else: 26 | out += '%s\n' % (file, file) 27 | 28 | out += '
' 32 | out += '' 33 | out += '' 34 | 35 | print 'Content-type: text/html' 36 | print 'Content-length: %d' % (len(out)) 37 | print '' 38 | print out 39 | -------------------------------------------------------------------------------- /rollcopy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # copy contents of: 3 | # to the directory install/rolls 4 | # usage: 5 | # rollcopy.sh 6 | # example: 7 | # rollcopy.sh /path/to/kernel.iso . 8 | # 9 | # @Copyright@ 10 | # @Copyright@ 11 | ROLLISO=$1 12 | DISTPATH=$2 13 | if [ ! -f $ROLLISO ]; then 14 | echo "$ROLLISO not found" 15 | exit -1; 16 | fi 17 | if [ ! -d /mnt/cdrom ]; then mkdir -p /mnt/cdrom; fi 18 | mount -o loop $ROLLISO /mnt/cdrom 19 | 20 | DESTPATH=$(readlink -f $DISTPATH) 21 | if [ ! -d $DESTPATH/install/rolls ]; then mkdir -p $DESTPATH/install/rolls; fi 22 | echo Copying "$ROLLISO" 23 | (cd /mnt/cdrom; tar cf - --exclude=TRANS.TBL --exclude=.discinfo --exclude=var --exclude=var --exclude=repodata --exclude=EFI --exclude=images --exclude=LiveOS --exclude=comps.xml --exclude=ks.cfg --exclude=isolinux .) | (cd $DESTPATH/install/rolls; tar xvfBp - ) 24 | 25 | umount /mnt/cdrom 26 | -------------------------------------------------------------------------------- /unpack-guides.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # unpack roll usersguide rpms found in 3 | # to /roll-documentation : 4 | # usage: 5 | # unpack-guides.sh 6 | # example: 7 | # unpack-guides.sh /path/to/install/rolls . 8 | # 9 | # @Copyright@ 10 | # @Copyright@ 11 | RPMPATH=$1 12 | DISTPATH=$2 13 | TMPPATH=$(mktemp -d) 14 | for guide in $(find $RPMPATH -name 'roll-*-usersguide*rpm'); do 15 | echo "found guide rpm: $guide" 16 | rpm2cpio $guide | ( cd $TMPPATH; cpio -id) 17 | done 18 | DESTPATH=$(readlink -f $DISTPATH) 19 | if [ ! -d $DESTPATH/roll-documentation ]; then mkdir -p $DESTPATH/roll-documentation; fi 20 | gpath=$(find $TMPPATH -name roll-documentation -prune) 21 | if [ "x$gpath" != "x" ]; then 22 | /bin/cp -fR $gpath/* $DESTPATH/roll-documentation 23 | fi 24 | /bin/rm -rf $TMPPATH 25 | 26 | --------------------------------------------------------------------------------