├── .gitignore ├── README.md └── cdk-v2 ├── .cdk ├── Vagrantfile └── tests ├── .rspec ├── Gemfile ├── Gemfile.lock ├── README.md ├── Rakefile └── spec ├── cdk ├── build_nodejs_spec.rb ├── image_push_spec.rb └── smoke │ ├── docker_spec.rb │ ├── openshift_spec.rb │ └── system_spec.rb └── spec_helper.rb /.gitignore: -------------------------------------------------------------------------------- 1 | */.vagrant 2 | **/images 3 | **/.ruby-* 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # All-in-One OpenShift Enterprise Vagrant VM 2 | 3 | 4 | 5 | - [What is it?](#what-is-it) 6 | - [Prerequisites](#prerequisites) 7 | - [How to run it](#how-to-run-it) 8 | - [How to access the VM's Docker daemon](#how-to-access-the-vms-docker-daemon) 9 | - [How to access the OpenShift registry](#how-to-access-the-openshift-registry) 10 | - [OpenShift Logins](#openshift-logins) 11 | - [Regular users](#regular-users) 12 | - [admin](#admin) 13 | - [Cluster admin](#cluster-admin) 14 | - [Known issues](#known-issues) 15 | - [Misc](#misc) 16 | - [How to use Landrush](#how-to-use-landrush) 17 | - [How to use Fabric8](#how-to-use-fabric8) 18 | - [How to use persistent volumes claims](#how-to-use-persistent-volumes-claims) 19 | - [How to run _any_ image on OpenShift](#how-to-run-_any_-image-on-openshift) 20 | - [How to sync an existing OpenShift project](#how-to-sync-an-existing-openshift-project) 21 | - [How to get HAProxy statistics](#how-to-get-haproxy-statistics) 22 | - [How to test webhooks locally](#how-to-test-webhooks-locally) 23 | - [How to debug EAP image](#how-to-debug-eap-image) 24 | - [How to find cause of container startup failure](#how-to-find-cause-of-container-startup-failure) 25 | - [How to explore the OpenShift REST API](#how-to-explore-the-openshift-rest-api) 26 | 27 | 28 | 29 | 30 | ## What is it? 31 | 32 | This repository contain a Vagrant setup to start a Vagrant virtual machine 33 | running a containerized version of OpenShift Enterprise 3.2 using CDK 2. 34 | 35 | 36 | ## Prerequisites 37 | 38 | The following prerequisites need to be met prior to creating and provisioning the 39 | virtual machine: 40 | 41 | * __[developers.redhat.com](http://developers.redhat.com) or Red Hat employee subscription credentials__ 42 | * __Active VPN connection during the creation and provisioning of the VM__ 43 | * [VirtualBox](https://www.virtualbox.org/) installed 44 | * [Vagrant](https://www.vagrantup.com/) installed 45 | * [vagrant-registration plugin](https://github.com/projectatomic/adb-vagrant-registration) _(>=1.2.1)_ installed 46 | * Run `vagrant plugin install vagrant-registration` to install plugin 47 | * [vagrant-service-manager plugin](https://github.com/projectatomic/vagrant-service-manager) _(>=1.1.0.beta.1)_ installed. 48 | * Run `vagrant plugin install vagrant-service-manager --plugin-version 1.1.0.beta.1` to install plugin 49 | * [vagrant-sshfs plugin](https://github.com/dustymabe/vagrant-sshfs) _(>=1.1.0)_ installed 50 | * Run `vagrant plugin install vagrant-sshfs` to install plugin 51 | * Optionally, the [OpenShift Client tools](https://github.com/openshift/origin/releases/) for your OS to run the `oc` commands from the terminal. 52 | * On Windows: 53 | * Ensure [Cygwin](https://www.cygwin.com/) is installed with rsync AND openssh. The default installation does not include these packages. 54 | 55 | 56 | ## How to run it 57 | 58 | $ cd cdk-v2 59 | $ export SUB_USERNAME= 60 | $ export SUB_PASSWORD= 61 | $ vagrant up 62 | 63 | This will start and provision the VM, as well as start an all-in-One OpenShift 64 | Enterprise instance. 65 | 66 | 67 | ## How to access the VM's Docker daemon 68 | 69 | Run the following command in your shell to configure it to use Docker (you need 70 | the Docker CLI binaries installed): 71 | 72 | ``` 73 | $ eval "$(vagrant service-manager env docker)" 74 | ``` 75 | 76 | 77 | ## How to access the OpenShift registry 78 | 79 | The OpenShift registry is per default exposed as _hub.openshift.10.1.2.2.xip.io_. You can 80 | push to this registry directly after logging in. Assuming one logs in as 81 | the defaultuser 'openshift-dev': 82 | 83 | $ oc login 10.1.2.2:8443 -u openshift-dev -p devel 84 | $ docker login -u openshift-dev -p `oc whoami -t` -e foo@bar.com hub.openshift.rhel-cdk.10.1.2.2.xip.io 85 | 86 | 87 | ## OpenShift Logins 88 | 89 | Once up an running the OpenShift console is accessible under https://10.1.2.2:8443/console/. 90 | 91 | 92 | ### Regular users 93 | 94 | The OpenShift instance setup with simple authentication. There is a _openshift-dev_ 95 | user with password _devel_ which can be used for creating projects and applications. 96 | 97 | 98 | ### admin 99 | 100 | There is also an _admin_ user who is member of the _cluster-admin_ group which 101 | has permissions to do everything on any project. 102 | 103 | 104 | ### Cluster admin 105 | 106 | To make any administrative changes to the system, you can also login to the VM (`vagrant ssh`) and use the command line tools with the _--config_ 107 | option referencing the _system:admin_ configuration. 108 | 109 | $ vagrant ssh 110 | $ oadm --config=/var/lib/origin/openshift.local.config/master/admin.kubeconfig 111 | $ oc --config=/var/lib/origin/openshift.local.config/master/admin.kubeconfig 112 | 113 | Alternatively you can set the _KUBECONFIG_ environment variable and skip the _--config_ option. 114 | 115 | $ export KUBECONFIG=/var/lib/origin/openshift.local.config/master/admin.kubeconfig 116 | 117 | However, be careful that when you in this case login as a different user, OpenShift 118 | will attempt to overwrite _admin.kubeconfig_. 119 | 120 | 121 | ## Known issues 122 | 123 | * Causes of failure on Windows 124 | * Ensure `VAGRANT_DETECTED_OS=cygwin` is set 125 | 126 | 127 | ## Misc 128 | 129 | 130 | ### How to use Landrush 131 | 132 | NOTE: Not working on Windows for now unless you build Landrush from source! 133 | 134 | Set the environment variable `OPENSHIFT_VAGRANT_USE_LANDRUSH` 135 | 136 | $ cd cdk-v2 137 | $ export SUB_USERNAME= 138 | $ export SUB_PASSWORD= 139 | $ export OPENSHIFT_VAGRANT_USE_LANDRUSH=true 140 | $ vagrant up 141 | 142 | The generated routes will then use _openshift.cdk_ as TLD instead of _10.1.2.2.xip.io_. 143 | 144 | 145 | ### How to use Fabric8 146 | 147 | Set the environment variable `INSTALL_FABRIC` 148 | 149 | $ cd cdk-v2 150 | $ export SUB_USERNAME= 151 | $ export SUB_PASSWORD= 152 | $ export INSTALL_FABRIC8=true 153 | $ vagrant up 154 | 155 | This will install and run [gofabric8](https://github.com/fabric8io/gofabric8/releases). 156 | After the Fabric8 pod is successfully running the Fabric8 console can be accessed 157 | via http://fabric8.openshift.10.1.2.2.xip.io/. Log in as admin and checkout 158 | the [Fabric8 Getting Started](http://fabric8.io/guide/getStarted/cdk.html) 159 | guide for your first steps. 160 | 161 | 162 | ### How to use persistent volumes claims 163 | 164 | The CDK provides a three persistent volumnes to experiment with. You can view them 165 | as admin as so: 166 | 167 | $ oc login -u admin -p password 168 | $ oc get pv 169 | NAME LABELS CAPACITY ACCESSMODES STATUS CLAIM REASON AGE 170 | pv01 1Gi RWO,RWX Available 6h 171 | pv02 2Gi RWO,RWX Available 6h 172 | pv03 3Gi RWO,RWX Available 6h 173 | 174 | To make a claim, you can do the following: 175 | 176 | # Using the openshift-dev user 177 | $ oc login -u openshift-dev -p devel 178 | 179 | # Using Nodejs as example app 180 | $ oc new-app https://github.com/openshift/nodejs-ex -l name=nodejs 181 | # Wait for build to complete ... 182 | $ oc expose service nodejs-ex -l name=nodejs 183 | 184 | # Make the persistent volume claim 185 | $ oc volume dc/nodejs-ex --add --claim-size 512M --mount-path /opt/app-root/src/views --name views 186 | persistentvolumeclaims/pvc-evu2v 187 | deploymentconfigs/nodejs-ex 188 | 189 | # Check the persistent volume claim 190 | oc get pvc 191 | NAME LABELS STATUS VOLUME CAPACITY ACCESSMODES AGE 192 | pvc-evu2v Bound pv01 1Gi RWO,RWX 1m 193 | 194 | # create some sample file 195 | $ echo '

It works!

' > /nfsvolumes/pv01/index.html 196 | 197 | # Brose to http://nodejs-ex-sample-project.rhel-cdk.10.1.2.2.xip.io/ 198 | 199 | # Verfify content on file system 200 | $ vagrant ssh 201 | $ ls -l /nfsvolumes/pv01 202 | 203 | # All app of the same app share the same volume 204 | $ oc scale dc/nodejs-ex --replicas 5 205 | $ oc get pods 206 | NAME READY STATUS RESTARTS AGE 207 | nodejs-ex-1-build 0/1 Completed 0 41m 208 | nodejs-ex-6-2hs75 1/1 Running 0 15m 209 | nodejs-ex-6-f576b 1/1 Running 0 1m 210 | nodejs-ex-6-fboe9 1/1 Running 0 1m 211 | nodejs-ex-6-ldaq1 1/1 Running 0 1m 212 | nodejs-ex-6-norrq 1/1 Running 0 1m 213 | 214 | $ oc exec -it nodejs-ex-6-norrq sh 215 | $ cat views/index.html 216 | 217 | 218 | ### How to run _any_ image on OpenShift 219 | 220 | Assuming user _openshift-dev_, you can do the following to run for example 221 | the Node.js based blogging framework [Ghost](https://ghost.org/). 222 | 223 | $ oc login 10.1.2.2:8443 -u openshift-dev -p devel 224 | Login successful. 225 | 226 | $ oc new-project my-ghost 227 | Now using project "my-ghost" on server "https://10.1.2.2:8443". 228 | 229 | $ docker pull ghost 230 | $ docker tag ghost hub.openshift.rhel-cdk.10.1.2.2.xip.io/my-ghost/ghost 231 | $ docker login -u openshift-dev -p `oc whoami -t` -e foo@bar.com hub.openshift.rhel-cdk.10.1.2.2.xip.io 232 | $ docker push hub.openshift.rhel-cdk.10.1.2.2.xip.io/my-ghost/ghost 233 | $ oc new-app --image-stream=ghost --name=ghost 234 | $ oc expose service ghost --hostname=my-ghost-blog.rhel-cdk.10.1.2.2.xip.io 235 | 236 | Then visit http://my-ghost-blog.rhel-cdk.10.1.2.2.xip.io/ with your browser. 237 | 238 | 239 | ### How to sync an existing OpenShift project 240 | 241 | First step is to export the configuration from the existing project: 242 | 243 | ``` 244 | $ oc export is,bc,dc,svc,route -o json > project-config.json 245 | ``` 246 | 247 | At this stage you probably want to edit the json and change the route. 248 | You can do this also after the import by `oc edit route`. 249 | 250 | Then on the second instance, create a new project, import the resources 251 | and trigger a new build: 252 | 253 | ``` 254 | $ oc new-project foo 255 | $ oc create -f project-config.json 256 | $ oc new-build 257 | ``` 258 | 259 | 260 | ### How to get HAProxy statistics 261 | 262 | The OpenShift HAProxy is configured to expose some statistics about the routes. 263 | This can sometimes be helpful when debugging problem or just to monitor traffic. 264 | To access the statistics use [http://10.1.2.2:1936/](http://10.1.2.2:1936). 265 | 266 | The username is '_admin_' and the password gets generated during the creation 267 | of the router pod. You can run the following to find the password: 268 | 269 | $ eval "$(vagrant service-manager env docker)" 270 | $ docker ps # You want the container id of the ose-haproxy-router container 271 | $ docker exec less /var/lib/haproxy/conf/haproxy.config | grep "stats auth" 272 | 273 | 274 | ### How to test webhooks locally 275 | 276 | Since the created VM is only visible on the host, GitHub webhooks won't work, since 277 | GitHub cannot reach the VM. Obviously you can just trigger the build via _oc_: 278 | 279 | ``` 280 | $ oc start-build 281 | ``` 282 | 283 | If you want to ensure that the actual webhooks work though, you can trigger them 284 | via curl as well. First determine the URLs of the GitHub and generic URL: 285 | 286 | ``` 287 | $ oc describe 288 | ``` 289 | 290 | To trigger the generic hook run: 291 | ``` 292 | $ curl -k -X POST 293 | ``` 294 | 295 | To trigger the GitHub hook run: 296 | ``` 297 | $ curl -k \ 298 | -H "Content-Type: application/json" \ 299 | -H "X-Github-Event: push" \ 300 | -X POST -d '{"ref":"refs/heads/master"}' \ 301 | 302 | ``` 303 | 304 | The GitHub payload is quite extensive, but the only thing which matters from 305 | an OpenShift perspective at the moment is that the _ref_ matches. 306 | 307 | 308 | ### How to debug EAP image 309 | 310 | This is based using the _jboss-eap-6/eap-openshift:6.4_ image from 311 | _registry.access.redhat.com_. This image is for example used by the _eap6-basic-sti_ 312 | template. 313 | 314 | The startup script _standalone.sh_ for the EAP instance within this image checks the 315 | variable _DEBUG_ to check whether to enable remote debugging on port 8787. 316 | 317 | ``` 318 | # Get the name of the deployment config. 319 | $ oc get dc 320 | NAME TRIGGERS LATEST VERSION 321 | eap-app ImageChange 1 322 | 323 | # Check the current environment variables (optional) 324 | $ oc env dc/eap-app --list 325 | OPENSHIFT_DNS_PING_SERVICE_NAME=eap-app-ping 326 | OPENSHIFT_DNS_PING_SERVICE_PORT=8888 327 | HORNETQ_CLUSTER_PASSWORD=mVxpNmqt 328 | HORNETQ_QUEUES= 329 | HORNETQ_TOPICS= 330 | 331 | # Set the DEBUG variable 332 | $ oc env dc/eap-app DEBUG=true 333 | 334 | # Double check the variable is set 335 | $oc env dc/eap-app --list 336 | OPENSHIFT_DNS_PING_SERVICE_NAME=eap-app-ping 337 | OPENSHIFT_DNS_PING_SERVICE_PORT=8888 338 | HORNETQ_CLUSTER_PASSWORD=mVxpNmqt 339 | HORNETQ_QUEUES= 340 | HORNETQ_TOPICS= 341 | DEBUG=true 342 | 343 | # Redeploy the latest image 344 | $ oc deploy eap-app --latest -n eap 345 | 346 | # Get the name of the running pod using the deployment config name as selector 347 | $ oc get pods -l deploymentConfig=eap-app 348 | NAME READY STATUS RESTARTS AGE 349 | eap-app-3-rw4ko 1/1 Running 0 1h 350 | 351 | # Port forward the debug port 352 | $ oc port-forward eap-app-3-rw4ko 8787:8787 353 | ``` 354 | 355 | Once the `oc port-forward` command is executed, you can attach a remote 356 | debugger to port 8787 on localhost. 357 | 358 | 359 | ### How to find cause of container startup failure 360 | 361 | In conjunction with trying to run arbitrary Docker images on OpenShift, it can be 362 | hard to track down deployment errors. If the deployment of a pot fails, OpenShift 363 | will try to reschedule a deployment and the original pod won't be available anymore. 364 | In this case you can try accessing the logs of the failing container directly 365 | via Docker commands against the Docker daemon running within the VM (the Docker 366 | daemon of the VM is used by the OpenShift instance itself as well). 367 | 368 | View the docker logs 369 | 370 | ``` 371 | $ vagrant ssh 372 | 373 | # Find the container id of the failing container (looking for the latest created container) 374 | $ docker ps -l -q 375 | 5b37abf17fb6 376 | 377 | $ docker logs 5b37abf17fb6 378 | ``` 379 | 380 | 381 | ### How to explore the OpenShift REST API 382 | 383 | Try this: 384 | * Open [this](http://openshift3swagger-claytondev.rhcloud.com) link in a browser 385 | * Paste the URL of the OpenShift instance "https://10.1.2.2:8443/swaggerapi/oapi/v1" into the input field 386 | -------------------------------------------------------------------------------- /cdk-v2/.cdk: -------------------------------------------------------------------------------- 1 | openshift.auth.scheme=Basic 2 | openshift.auth.username=openshift-dev 3 | openshift.auth.password=devel 4 | cdk.version=2.0 5 | -------------------------------------------------------------------------------- /cdk-v2/Vagrantfile: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | # Usage: 5 | # 6 | # To use this Vagrantfile: 7 | # * Have valid RH employee subscription account 8 | # * Be connected to the internal Red Hat network 9 | # * Have the 'vagrant-registration' plugin installed 10 | # * Have the 'vagrant-service-manager' plugin installed 11 | 12 | # General configuration: 13 | 14 | # URLs from where to fetch the Vagrant VirtualBox images 15 | # FIXME: How to point to the official box image? 16 | VAGRANT_VIRTUALBOX_URL='http://cdk-builds.usersys.redhat.com/builds/nightly/27-May-2016/rhel-7-cdk-vagrant-scratch-7.2.27052016-1.x86_64.vagrant-virtualbox.box' 17 | VAGRANT_LIBVIRT_URL='http://cdk-builds.usersys.redhat.com/builds/nightly/27-May-2016/rhel-7-cdk-vagrant-scratch-7.2.27052016-1.x86_64.vagrant-libvirt.box' 18 | 19 | # The IP address and hostname of the VM as exposed on the host. 20 | # Uses xip.io for now to have zero configuration routes 21 | PUBLIC_ADDRESS="10.1.2.2" 22 | REGISTRY_HOST="hub.openshift.#{PUBLIC_ADDRESS}.xip.io" 23 | 24 | # The amount of memory available to the VM 25 | VM_MEMORY = ENV['VM_MEMORY'] || 3072 26 | 27 | # Number of VM CPUs 28 | VM_CPU = ENV['VM_CPU'] || 2 29 | 30 | # The Docker registry from where we pull the OpenShift Docker image 31 | DOCKER_REGISTRY="brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888" 32 | 33 | # The name of the OpenShift image. 34 | IMAGE_NAME="openshift3/ose" 35 | IMAGE_TAG="v3.2.0.8" 36 | 37 | SUBSCRIPTION_INFO = " 38 | Red Hat subscription credentials are needed for this VM. 39 | You can supply them interactively or by setting environment variables. 40 | Set these environment variables to your subscription username/password to avoid interactive registration: 41 | 42 | $ export SUB_USERNAME=rhn-username 43 | $ export SUB_PASSWORD=password 44 | " 45 | 46 | REQUIRED_PLUGINS = %w(vagrant-service-manager vagrant-registration vagrant-sshfs) 47 | message = ->(name) { "#{name} plugin is not installed, run `vagrant plugin install #{name}` to install it." } 48 | errors = [] 49 | # Validate and collect error message if plugin is not installed 50 | REQUIRED_PLUGINS.each {|plugin| errors << message.call(plugin) unless Vagrant.has_plugin?(plugin) } 51 | msg = errors.size > 1 ? "Errors: \n* #{errors.join("\n* ")}" : "Error: #{errors.first}" 52 | raise Vagrant::Errors::VagrantError.new, msg unless errors.size == 0 53 | 54 | Vagrant.configure(2) do |config| 55 | 56 | config.vm.define "cdk" do |cdk| 57 | cdk.vm.provider "virtualbox" do |v, override| 58 | override.vm.box = "cdk_v2" 59 | override.vm.box_url = "#{VAGRANT_VIRTUALBOX_URL}" 60 | v.name = "openshift.cdk-2" 61 | v.memory = VM_MEMORY 62 | v.cpus = VM_CPU 63 | v.customize ["modifyvm", :id, "--ioapic", "on"] 64 | v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"] 65 | end 66 | 67 | cdk.vm.provider "libvirt" do |v, override| 68 | override.vm.box = "cdk_v2" 69 | override.vm.box_url = "#{VAGRANT_LIBVIRT_URL}" 70 | v.driver = "kvm" 71 | v.memory = VM_MEMORY 72 | v.cpus = VM_CPU 73 | end 74 | 75 | cdk.vm.network "private_network", ip: "#{PUBLIC_ADDRESS}" 76 | 77 | if ENV.has_key?('SUB_USERNAME') && ENV.has_key?('SUB_PASSWORD') 78 | cdk.registration.username = ENV['SUB_USERNAME'] 79 | cdk.registration.password = ENV['SUB_PASSWORD'] 80 | # After provisioning registration can be skipped as well as un-registering on halt 81 | #cdk.registration.skip = true 82 | #cdk.registration.unregister_on_halt = false 83 | end 84 | 85 | case ARGV[0] 86 | when "up", "halt" 87 | if cdk.registration.username.nil? || cdk.registration.password.nil? 88 | puts SUBSCRIPTION_INFO 89 | end 90 | end 91 | 92 | cdk.vm.synced_folder '.', '/vagrant', disabled: true 93 | if Vagrant::Util::Platform.windows? 94 | target_path = ENV['USERPROFILE'].gsub(/\\/,'/').gsub(/[[:alpha:]]{1}:/){|s|'/' + s.downcase.sub(':', '')} 95 | cdk.vm.synced_folder ENV['USERPROFILE'], target_path, type: 'sshfs', sshfs_opts_append: '-o umask=000 -o uid=1000 -o gid=1000' 96 | else 97 | cdk.vm.synced_folder ENV['HOME'], ENV['HOME'], type: 'sshfs', sshfs_opts_append: '-o umask=000 -o uid=1000 -o gid=1000' 98 | end 99 | cdk.vm.provision "shell", inline: <<-SHELL 100 | sudo setsebool -P virt_sandbox_use_fusefs 1 101 | SHELL 102 | 103 | # Set OPENSHIFT_VAGRANT_USE_LANDRUSH=true in your shell in order to use Landrush 104 | if Vagrant.has_plugin?('landrush') && ENV.fetch('OPENSHIFT_VAGRANT_USE_LANDRUSH', 'false').to_s.downcase == 'true' 105 | cdk.landrush.enabled = true 106 | tld = 'openshift.cdk' 107 | cdk.landrush.tld = tld 108 | cdk.landrush.host tld, "#{PUBLIC_ADDRESS}" 109 | cdk.vm.provision :shell, inline: <<-SHELL 110 | sed -i.orig -e "s/OPENSHIFT_SUBDOMAIN=.*/OPENSHIFT_SUBDOMAIN='#{tld}'/g" /etc/sysconfig/openshift_option 111 | SHELL 112 | end 113 | 114 | if ENV.fetch('INSTALL_FABRIC8', 'false').to_s.downcase == 'true' 115 | cdk.vm.provision "shell", inline: <<-SHELL 116 | mkdir fabric8 && cd fabric8 117 | curl -L -O https://github.com/fabric8io/gofabric8/releases/download/v0.4.27/gofabric8-0.4.27-linux-amd64.tar.gz 118 | tar -xvzf gofabric8-0.4.27-linux-amd64.tar.gz 119 | oc login --insecure-skip-tls-verify=true #{PUBLIC_ADDRESS}:8443 -u=admin -p=admin 120 | ./gofabric8 deploy -y --domain=openshift.#{PUBLIC_ADDRESS}.xip.io 121 | ./gofabric8 secrets -y 122 | SHELL 123 | end 124 | 125 | cdk.vm.provision "shell", inline: <<-SHELL 126 | echo 127 | echo "Successfully started and provisioned VM with #{VM_CPU} cores and #{VM_MEMORY} MB of memory." 128 | echo "To modify the number of cores and/or available memory set the environment variables" 129 | echo "VM_CPU respectively VM_MEMORY." 130 | echo 131 | echo "You can now access the OpenShift console on: https://#{PUBLIC_ADDRESS}:8443/console" 132 | echo 133 | echo "To use OpenShift CLI, run:" 134 | echo "$ vagrant ssh" 135 | echo "$ oc login #{PUBLIC_ADDRESS}:8443" 136 | echo 137 | echo "Configured users are (/):" 138 | echo "openshift-dev/devel" 139 | echo "admin/admin" 140 | echo 141 | echo "If you have the oc client library on your host, you can also login from your host." 142 | echo 143 | SHELL 144 | end 145 | end 146 | -------------------------------------------------------------------------------- /cdk-v2/tests/.rspec: -------------------------------------------------------------------------------- 1 | --color 2 | --format documentation 3 | -------------------------------------------------------------------------------- /cdk-v2/tests/Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | gem 'serverspec' 4 | gem 'rake' 5 | -------------------------------------------------------------------------------- /cdk-v2/tests/Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | diff-lcs (1.2.5) 5 | multi_json (1.11.2) 6 | net-scp (1.2.1) 7 | net-ssh (>= 2.6.5) 8 | net-ssh (3.0.2) 9 | net-telnet (0.1.1) 10 | rake (10.5.0) 11 | rspec (3.4.0) 12 | rspec-core (~> 3.4.0) 13 | rspec-expectations (~> 3.4.0) 14 | rspec-mocks (~> 3.4.0) 15 | rspec-core (3.4.3) 16 | rspec-support (~> 3.4.0) 17 | rspec-expectations (3.4.0) 18 | diff-lcs (>= 1.2.0, < 2.0) 19 | rspec-support (~> 3.4.0) 20 | rspec-its (1.2.0) 21 | rspec-core (>= 3.0.0) 22 | rspec-expectations (>= 3.0.0) 23 | rspec-mocks (3.4.1) 24 | diff-lcs (>= 1.2.0, < 2.0) 25 | rspec-support (~> 3.4.0) 26 | rspec-support (3.4.1) 27 | serverspec (2.30.0) 28 | multi_json 29 | rspec (~> 3.0) 30 | rspec-its 31 | specinfra (~> 2.48) 32 | sfl (2.2) 33 | specinfra (2.52.0) 34 | net-scp 35 | net-ssh (>= 2.7, < 3.1) 36 | net-telnet 37 | sfl 38 | 39 | PLATFORMS 40 | ruby 41 | 42 | DEPENDENCIES 43 | rake 44 | serverspec 45 | -------------------------------------------------------------------------------- /cdk-v2/tests/README.md: -------------------------------------------------------------------------------- 1 | # Serverspec test suite for Vagrant OpenShift setup 2 | 3 | 4 | 5 | - [Getting started](#getting-started) 6 | - [Running tests](#running-tests) 7 | - [Tips](#tips) 8 | 9 | 10 | 11 | This directory containts [Serverspec](http://serverspec.org/) tests to verify 12 | the Vagrant OpenShift setup works and to document the expected behavior 13 | 14 | 15 | ## Getting started 16 | 17 | * Install a Ruby (2.2 is verified to work) 18 | * Install Bundler - `gem install bundler --version 1.10.5` 19 | * Install test harness dependencies - `bundle install` 20 | 21 | 22 | ## Running tests 23 | 24 | * `bundle exec rake -T` - view available rake tasks 25 | * `bundle exec rake` - run all tests 26 | * `bundle exec rake spec:cdk-all` - explicitly run all tests 27 | * `bundle exec rake spec:cdk-smoke` - run smoke tests only 28 | * `rake spec:cdk-smoke[10.10.10.2]` (or whatever the IP us you selected 29 | in the _Vagrantfile_) - run tests against a different IP 30 | 31 | 32 | ## Tips 33 | 34 | * You can run any `vagrant` command directly from the _test_ directory. 35 | Vagrant climbs up the directory tree looking for the first Vagrantfile it can find. 36 | See also - [Vagrantfile docs](https://www.vagrantup.com/docs/vagrantfile/). 37 | -------------------------------------------------------------------------------- /cdk-v2/tests/Rakefile: -------------------------------------------------------------------------------- 1 | require 'rake' 2 | require 'rspec/core/rake_task' 3 | 4 | task :spec => 'spec:all' 5 | task :default => :spec 6 | 7 | namespace :spec do 8 | targets = [] 9 | Dir.glob('./spec/*').each do |dir| 10 | next unless File.directory?(dir) 11 | targets << File.basename(dir) 12 | end 13 | 14 | task :all => targets.map{|target| target + '-all'} 15 | task :default => :all 16 | 17 | targets.each do |target| 18 | desc "Run all serverspec tests against #{target}" 19 | RSpec::Core::RakeTask.new((target + '-all').to_sym, :ip) do |t, args| 20 | ENV['TARGET_HOST'] = target 21 | t.pattern = "spec/#{target}/**/*_spec.rb" 22 | 23 | ip = args[:ip] 24 | if ip.nil? 25 | ip = '10.1.2.2' 26 | end 27 | ENV['TARGET_IP'] = ip 28 | puts "Target Host: #{ENV['TARGET_HOST']}" 29 | puts "Target IP: #{ENV['TARGET_IP']} " 30 | end 31 | end 32 | 33 | targets.each do |target| 34 | desc "Run all smoke serverspec tests against #{target}" 35 | RSpec::Core::RakeTask.new((target + '-smoke').to_sym, :ip) do |t, args| 36 | ENV['TARGET_HOST'] = target 37 | t.pattern = "spec/#{target}/smoke/*_spec.rb" 38 | 39 | ip = args[:ip] 40 | if ip.nil? 41 | ip = '10.1.2.2' 42 | end 43 | ENV['TARGET_IP'] = ip 44 | puts "Target Host: #{ENV['TARGET_HOST']}" 45 | puts "Target IP: #{ENV['TARGET_IP']}" 46 | end 47 | end 48 | 49 | targets.each do |target| 50 | Dir.glob("spec/#{target}/**/*_spec.rb").each do |spec| 51 | spec_name = File.basename(spec).chomp('_spec.rb') 52 | desc "Run single spec #{spec_name} against #{target}" 53 | RSpec::Core::RakeTask.new((target + '-' + spec_name).to_sym, :ip) do |t, args| 54 | ENV['TARGET_HOST'] = target 55 | t.pattern = spec 56 | 57 | ip = args[:ip] 58 | if ip.nil? 59 | ip = '10.1.2.2' 60 | end 61 | ENV['TARGET_IP'] = ip 62 | puts "Target Host: #{ENV['TARGET_HOST']}" 63 | puts "Target IP: #{ENV['TARGET_IP']}" 64 | end 65 | end 66 | end 67 | end 68 | -------------------------------------------------------------------------------- /cdk-v2/tests/spec/cdk/build_nodejs_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | ############################################################################### 4 | # Tests verifying pushing an arbitrary image 5 | ############################################################################### 6 | 7 | describe "Nodejs example app" do 8 | let(:disable_sudo) { false } 9 | it "should build" do 10 | # Login 11 | command("oc --insecure-skip-tls-verify login #{ENV['TARGET_IP']}:8443 -u openshift-dev -p devel").exit_status.should be 0 12 | 13 | # Create a new project 14 | command('oc new-project nodejs').exit_status.should be 0 15 | 16 | # Create the app 17 | command('oc new-app nodejs-example').exit_status.should be 0 18 | sleep 5 19 | 20 | i = 0 21 | while i < 60 22 | state = command('oc get pods --no-headers').stdout.split[2] 23 | case state 24 | when "Pending" 25 | puts 'Pulling builder image' 26 | sleep 10 27 | when "Running" 28 | puts 'Building app' 29 | sleep 10 30 | when "Completed" 31 | # TODO give time to get the pod up, needs improvement 32 | puts 'Build complete. Waiting for pod to start' 33 | sleep 10 34 | break 35 | else 36 | fail "Unexpected builder pod state: #{state}" 37 | end 38 | i = i+1 39 | end 40 | 41 | # Verify Nodejs app is up and running 42 | uri = URI.parse("http://nodejs-example-nodejs.rhel-cdk.#{ENV['TARGET_IP']}.xip.io") 43 | http = Net::HTTP.new(uri.host, uri.port) 44 | request = Net::HTTP::Get.new(uri.request_uri) 45 | response = http.request(request) 46 | response.code.should match /200/ 47 | response.body.should match /Welcome to your Node.js application on OpenShift/ 48 | end 49 | 50 | after do 51 | command('oc delete all --all').exit_status.should be 0 52 | command('oc delete project nodejs').exit_status.should be 0 53 | command('oc logout').exit_status.should be 0 54 | end 55 | end 56 | 57 | -------------------------------------------------------------------------------- /cdk-v2/tests/spec/cdk/image_push_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | ############################################################################### 4 | # Tests verifying pushing an arbitrary image 5 | ############################################################################### 6 | 7 | describe "Pushing arbitrary docker image" do 8 | let(:disable_sudo) { false } 9 | it "should succeed" do 10 | # Using Ghost as image to test. Pulled from Docker Hub 11 | command('docker pull ghost').exit_status.should be 0 12 | 13 | # We need a project to "host" our image 14 | command("oc --insecure-skip-tls-verify login #{ENV['TARGET_IP']}:8443 -u openshift-dev -p devel").exit_status.should be 0 15 | command('oc new-project myproject').exit_status.should be 0 16 | token = command('oc whoami -t').stdout 17 | 18 | # 1 - Tag the image against the exposed OpenShift registry using the created project name as target 19 | # 2 - Log into the Docker registry 20 | # 3 - Push the image 21 | command("docker tag -f ghost hub.openshift.rhel-cdk.#{ENV['TARGET_IP']}.xip.io/myproject/ghost").exit_status.should be 0 22 | command("docker login -u openshift-dev -p '#{token}' -e foo@bar.com hub.openshift.rhel-cdk.#{ENV['TARGET_IP']}.xip.io").exit_status.should be 0 23 | command("docker push hub.openshift.rhel-cdk.#{ENV['TARGET_IP']}.xip.io/myproject/ghost").exit_status.should be 0 24 | 25 | # 1 - Create the app from the image stream created by pusing the image 26 | # 2 - Expose a route to the app 27 | command('oc new-app --image-stream=ghost --name=ghost').exit_status.should be 0 28 | command("oc expose service ghost --hostname=ghost.#{ENV['TARGET_IP']}.xip.io").exit_status.should be 0 29 | # TODO - instead of sleep we should use oc to monitor the state of the pod until running 30 | sleep 60 31 | 32 | # Verify Ghost is up and running 33 | uri = URI.parse("http://ghost.#{ENV['TARGET_IP']}.xip.io") 34 | http = Net::HTTP.new(uri.host, uri.port) 35 | request = Net::HTTP::Get.new(uri.request_uri) 36 | response = http.request(request) 37 | response.code.should match /200/ 38 | end 39 | 40 | after do 41 | command("docker rmi hub.openshift.rhel-cdk.#{ENV['TARGET_IP']}.xip.io/myproject/ghost").exit_status.should be 0 42 | command('oc delete all --all').exit_status.should be 0 43 | command('oc delete project myproject').exit_status.should be 0 44 | command('oc logout').exit_status.should be 0 45 | command('rm /home/vagrant/.docker/config.json').exit_status.should be 0 46 | end 47 | end 48 | 49 | -------------------------------------------------------------------------------- /cdk-v2/tests/spec/cdk/smoke/docker_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | ############################################################################### 4 | # Tests verifying everything Docker 5 | ############################################################################### 6 | describe service('docker') do 7 | it { should be_enabled } 8 | end 9 | 10 | describe service('docker') do 11 | it { should be_running } 12 | end 13 | 14 | describe file('/etc/sysconfig/docker') do 15 | it { should contain 'registry.access.redhat.com' } 16 | end 17 | 18 | describe docker_container('openshift') do 19 | it { should be_running } 20 | end 21 | 22 | describe command('docker ps --filter "name=k8s_registry.*"') do 23 | its(:stdout) { should match /ose-docker-registry/ } 24 | end 25 | 26 | describe command('docker ps --filter "name=k8s_router.*"') do 27 | its(:stdout) { should match /ose-haproxy-router/ } 28 | end 29 | 30 | describe command('docker pull alpine') do 31 | its(:exit_status) { should eq 0 } 32 | end 33 | 34 | describe docker_image('alpine:latest') do 35 | it { should exist } 36 | end 37 | 38 | describe x509_certificate('/etc/docker/server-cert.pem') do 39 | let(:disable_sudo) { false } 40 | it { should be_valid } 41 | end 42 | 43 | describe command('openssl x509 -text -noout -in /etc/docker/server-cert.pem') do 44 | let(:disable_sudo) { false } 45 | its(:stdout) { should match /IP Address:#{Regexp.quote(ENV['TARGET_IP'])}/ } 46 | end 47 | 48 | 49 | -------------------------------------------------------------------------------- /cdk-v2/tests/spec/cdk/smoke/openshift_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | require 'net/http' 3 | require 'uri' 4 | 5 | ############################################################################### 6 | # Tests verifying the correct installation of OpenShift 7 | ############################################################################### 8 | 9 | describe service('openshift') do 10 | it { should be_running } 11 | end 12 | 13 | describe port(8443) do 14 | let(:disable_sudo) { false } 15 | it { should be_listening } 16 | end 17 | 18 | describe command("curl -k https://#{ENV['TARGET_IP']}:8443/console/") do 19 | its(:stdout) { should contain /OpenShift Web Console/ } 20 | end 21 | 22 | describe command("oc --insecure-skip-tls-verify login #{ENV['TARGET_IP']}:8443 -u openshift-dev -p devel") do 23 | its(:stdout) { should contain /Login successful./ } 24 | end 25 | 26 | describe command("oc --insecure-skip-tls-verify login #{ENV['TARGET_IP']}:8443 -u admin -p admin") do 27 | its(:stdout) { should contain /Login successful./ } 28 | end 29 | 30 | describe "OpenShift registry route" do 31 | it "should be exposed" do 32 | registry_get = command('sudo oc --config=/var/lib/openshift/openshift.local.config/master/admin.kubeconfig get route/docker-registry').stdout 33 | registry_get.should contain /hub.openshift.rhel-cdk.#{Regexp.quote(ENV['TARGET_IP'])}.xip.io/ 34 | end 35 | end 36 | 37 | describe "Login to OpenShift registry via exposed registry route hub.openshift.rhel-cdk.#{ENV['TARGET_IP']}.xip.io" do 38 | it "should succeed" do 39 | exit = command("oc --insecure-skip-tls-verify login #{ENV['TARGET_IP']}:8443 -u openshift-dev -p devel").exit_status 40 | exit.should be 0 41 | token = command('oc whoami -t').stdout 42 | exit = command("docker login -u openshift-dev -p '#{token}' -e foo@bar.com hub.openshift.rhel-cdk.#{ENV['TARGET_IP']}.xip.io").exit_status 43 | exit.should be 0 44 | end 45 | 46 | after do 47 | command('oc logout').exit_status.should be 0 48 | command('rm /home/vagrant/.docker/config.json').exit_status.should be 0 49 | end 50 | end 51 | 52 | describe "Admin user" do 53 | it "should be able to list OpenShift nodes" do 54 | command("oc --insecure-skip-tls-verify login #{ENV['TARGET_IP']}:8443 -u admin -p admin").exit_status.should be 0 55 | nodes = command('oc get nodes').stdout 56 | nodes.should contain /rhel-cdk/ 57 | end 58 | 59 | after do 60 | command('oc logout').exit_status.should be 0 61 | end 62 | end 63 | 64 | describe "Basic application templates" do 65 | it "should exist" do 66 | command("oc --insecure-skip-tls-verify login #{ENV['TARGET_IP']}:8443 -u admin -p admin").exit_status.should be 0 67 | templates = command('oc --insecure-skip-tls-verify get templates -n openshift').stdout 68 | # TODO - complete list after requirements are set 69 | templates.should contain /eap64-basic-s2i/ 70 | templates.should contain /eap64-mysql-persistent-s2i/ 71 | templates.should contain /nodejs-example/ 72 | end 73 | 74 | after do 75 | command('oc logout').exit_status.should be 0 76 | end 77 | end 78 | 79 | describe "OpenShift health URL" do 80 | it "should respond with response code 200" do 81 | uri = URI.parse("https://#{ENV['TARGET_IP']}:8443/healthz/ready") 82 | http = Net::HTTP.new(uri.host, uri.port) 83 | http.use_ssl = true 84 | http.verify_mode = OpenSSL::SSL::VERIFY_NONE 85 | request = Net::HTTP::Get.new(uri.request_uri) 86 | response = http.request(request) 87 | response.code.should match /200/ 88 | end 89 | end 90 | 91 | describe file('/usr/bin/oc') do 92 | it { should be_file } 93 | it { should be_executable } 94 | end 95 | 96 | describe file('/usr/bin/oadm') do 97 | it { should be_symlink } 98 | it { should be_linked_to '/usr/bin/openshift' } 99 | it { should be_executable } 100 | end 101 | 102 | -------------------------------------------------------------------------------- /cdk-v2/tests/spec/cdk/smoke/system_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper' 2 | 3 | ############################################################################### 4 | # Tests verifying system state and services 5 | ############################################################################### 6 | describe interface('eth0') do 7 | it { should be_up } 8 | end 9 | 10 | describe interface('eth0') do 11 | let(:disable_sudo) { false } 12 | it { should have_ipv4_address("10.0.2.15") } 13 | end 14 | 15 | describe interface('eth1') do 16 | it { should be_up } 17 | end 18 | 19 | describe interface('eth1') do 20 | let(:disable_sudo) { false } 21 | it { should have_ipv4_address("#{ENV['TARGET_IP']}") } 22 | end 23 | 24 | describe interface('docker0') do 25 | it { should be_up } 26 | end 27 | 28 | describe file(ENV['HOME']) do 29 | it { should be_directory } 30 | end 31 | 32 | -------------------------------------------------------------------------------- /cdk-v2/tests/spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | require 'serverspec' 2 | require 'net/ssh' 3 | require 'tempfile' 4 | 5 | def which(cmd) 6 | exts = ENV['PATHEXT'] ? ENV['PATHEXT'].split(';') : [''] 7 | ENV['PATH'].split(File::PATH_SEPARATOR).each do |path| 8 | exts.each { |ext| 9 | exe = File.join(path, "#{cmd}#{ext}") 10 | return exe if File.executable?(exe) && !File.directory?(exe) 11 | } 12 | end 13 | return nil 14 | end 15 | 16 | set :backend, :ssh 17 | 18 | if ENV['ASK_SUDO_PASSWORD'] 19 | begin 20 | require 'highline/import' 21 | rescue LoadError 22 | fail "highline is not available. Try installing it." 23 | end 24 | set :sudo_password, ask("Enter sudo password: ") { |q| q.echo = false } 25 | else 26 | set :sudo_password, ENV['SUDO_PASSWORD'] 27 | end 28 | 29 | host = ENV['TARGET_HOST'] 30 | 31 | `vagrant up #{host}` 32 | 33 | config = Tempfile.new('', Dir.tmpdir) 34 | `vagrant ssh-config #{host} > #{config.path}` 35 | 36 | options = Net::SSH::Config.for(host, [config.path]) 37 | 38 | options[:user] ||= Etc.getlogin 39 | 40 | set :host, options[:host_name] || host 41 | set :ssh_options, options 42 | 43 | # Disable sudo 44 | set :disable_sudo, true 45 | 46 | 47 | # Set environment variables 48 | # set :env, :LANG => 'C', :LC_MESSAGES => 'C' 49 | 50 | # Set PATH 51 | # set :path, '/sbin:/usr/local/sbin:$PATH' 52 | --------------------------------------------------------------------------------