├── README.md ├── chapter-01 └── identity-wf │ ├── pom.xml │ └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml └── chapter-04 ├── distcp-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── email-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── fs-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── hive-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── java-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── map-reduce-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── mr-streaming-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── pig-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── shell-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── sqoop-eval-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── sqoop-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml ├── ssh-wf ├── pom.xml └── src │ └── main │ ├── assemblies │ └── example.xml │ ├── client │ └── job.properties │ ├── data │ └── input │ │ └── input.txt │ └── xml │ └── workflow.xml └── subworkflow-wf ├── pom.xml └── src └── main ├── assemblies └── example.xml ├── client └── job.properties ├── data └── input │ └── input.txt └── xml └── workflow.xml /README.md: -------------------------------------------------------------------------------- 1 | This repository is still a work in progress. 2 | 3 | This repository contains code examples for the book "Apache Oozie" by Mohammad Kamrul Islam and Aravind Srinivasan (O'Reilly). 4 | 5 | # How to build the examples 6 | 7 | Clone this repository on your local machine, move to the project root and run `mvn clean assembly:single`. 8 | 9 | Example for chapter 1: 10 | 11 | ``` 12 | $ git clone https://github.com/oozie-book/examples.git 13 | $ cd examples/chapter-01/identity-wf 14 | $ mvn clean assembly:single 15 | $ tree target/example 16 | target/example 17 | |-- ch01-identity 18 | | |-- app 19 | | | `-- workflow.xml 20 | | `-- data 21 | | `-- input 22 | | `-- input.txt 23 | `-- job.properties 24 | 25 | 4 directories, 3 files 26 | ``` 27 | -------------------------------------------------------------------------------- /chapter-01/identity-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch01-identity 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 01 - identity 27 | Oozie book example, ch01 - identity 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-01/identity-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-01/identity-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | nameNode=hdfs://localhost:8020 20 | jobTracker=localhost:8021 21 | exampleDir=${nameNode}/user/${user.name}/ch01-identity 22 | 23 | oozie.wf.application.path=${exampleDir}/app 24 | -------------------------------------------------------------------------------- /chapter-01/identity-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-01/identity-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | exampleDir 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | ${jobTracker} 37 | ${nameNode} 38 | 39 | 40 | 41 | 42 | 43 | mapred.mapper.class 44 | org.apache.hadoop.mapred.lib.IdentityMapper 45 | 46 | 47 | mapred.reducer.class 48 | org.apache.hadoop.mapred.lib.IdentityReducer 49 | 50 | 51 | mapred.input.dir 52 | ${exampleDir}/data/input 53 | 54 | 55 | mapred.output.dir 56 | ${exampleDir}/data/output 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | The Identity Map-Reduce job failed! 66 | 67 | 68 | 69 | 70 | 71 | -------------------------------------------------------------------------------- /chapter-04/distcp-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-distcp-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - Hadoop distcp action 27 | Oozie book example, ch04 - Hadoop distcp action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/distcp-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/distcp-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-distcp-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/distcp-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/distcp-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | 36 | 37 | 38 | -Dfs.s3n.awsAccessKeyId=XXXX 39 | -Dfs.s3n.awsSecretAccessKey=YYYY 40 | -m 41 | 100 42 | s3n://my-logfiles/2014-04-15/* 43 | /hdfs/user/joe/logs/2014-04-15/ 44 | 45 | 46 | 47 | 48 | 49 | 50 | The example Hadoop distcp action failed! 51 | 52 | 53 | 54 | 55 | 56 | 57 | -------------------------------------------------------------------------------- /chapter-04/email-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-email-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - E-mail action 27 | Oozie book example, ch04 - E-mail action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/email-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/email-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-email-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/email-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/email-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | joe@initech.com,the_other_joe@initech.com 34 | john@initech.com 35 | Email notifications for ${wf:id()} 36 | The wf ${wf:id()} successfully completed. 37 | 38 | 39 | 40 | 41 | 42 | 43 | The example e-mail action failed! 44 | 45 | 46 | 47 | 48 | 49 | 50 | -------------------------------------------------------------------------------- /chapter-04/fs-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-fs-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - FS action 27 | Oozie book example, ch04 - FS action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/fs-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/fs-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-fs-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/fs-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/fs-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | hdfs://nn.mycompany.com:8020 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | The example FS action failed! 44 | 45 | 46 | 47 | 48 | 49 | 50 | -------------------------------------------------------------------------------- /chapter-04/hive-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-hive-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - Hive action 27 | Oozie book example, ch04 - Hive action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/hive-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/hive-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-hive-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/hive-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/hive-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | hive-config.xml 36 | 37 | -hivevar 38 | age=30 39 | 40 | 41 | 42 | 43 | 44 | 45 | The example Hive action failed! 46 | 47 | 48 | 49 | 50 | 51 | 52 | -------------------------------------------------------------------------------- /chapter-04/java-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-java-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - java action 27 | Oozie book example, ch04 - java action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/java-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/java-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-java-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/java-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/java-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | 36 | 37 | 38 | myAppClass 39 | -D 40 | mapreduce.reduce.memory.mb=8192 41 | hdfs://nn.mycompany.com:8020/hdfs/user/joe/input 42 | hdfs://nn.mycompany.com:8020/hdfs/user/joe/output 43 | prod 44 | myApp.jar#myApp.jar 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | The example Java action failed! 53 | 54 | 55 | 56 | 57 | 58 | -------------------------------------------------------------------------------- /chapter-04/map-reduce-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-map-reduce-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - map-reduce action 27 | Oozie book example, ch04 - map-reduce action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/map-reduce-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/map-reduce-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-map-reduce-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/map-reduce-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/map-reduce-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | 36 | 37 | 38 | 39 | 40 | mapred.mapper.class 41 | com.myBiz.mr.MyMapClass 42 | 43 | 44 | mapred.reducer.class 45 | com.myBiz.mr.MyRedClass 46 | 47 | 48 | mapred.job.reduce.memory.mb 49 | 8192 50 | 51 | 52 | mapred.input.dir 53 | /hdfs/user/joe/input 54 | 55 | 56 | mapred.output.dir 57 | /hdfs/user/joe/output 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | The example Map-Reduce job failed! 67 | 68 | 69 | 70 | 71 | 72 | -------------------------------------------------------------------------------- /chapter-04/mr-streaming-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-mr-streaming-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - mr streamng action 27 | Oozie book example, ch04 - mr streaming action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/mr-streaming-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/mr-streaming-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-mr-streaming-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/mr-streaming-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/mr-streaming-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | 36 | 37 | 38 | 39 | python mapper.py 40 | python reducer.py 41 | 42 | 43 | 44 | mapred.input.dir 45 | /hdfs/user/joe/input 46 | 47 | 48 | mapred.output.dir 49 | /hdfs/user/joe/output 50 | 51 | 52 | wfDir/mapper.py#mapper.py 53 | wfDir/redcer.py#reducer.py 54 | 55 | 56 | 57 | 58 | 59 | 60 | The example Streaming MR job failed! 61 | 62 | 63 | 64 | 65 | 66 | -------------------------------------------------------------------------------- /chapter-04/pig-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-pig-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - Pig action 27 | Oozie book example, ch04 - Pig action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/pig-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/pig-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-pig-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/pig-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/pig-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | 36 | 37 | 38 | 39 | 40 | mapred.job.queue.name 41 | research 42 | 43 | 44 | 45 | -param 46 | age=30 47 | -param 48 | output=hdfs://nn.mycompany.com:8020/hdfs/user/joe/pig/output 49 | 50 | 51 | 52 | 53 | 54 | 55 | The example Pig action failed! 56 | 57 | 58 | 59 | 60 | 61 | 62 | -------------------------------------------------------------------------------- /chapter-04/shell-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-shell-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - Shell action 27 | Oozie book example, ch04 - Shell action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/shell-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/shell-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-shell-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/shell-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/shell-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | /usr/bin/python 36 | test.py 37 | 07/21/2014 38 | TZ=PST 39 | test.py#test.py 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | The example shell action failed! 48 | 49 | 50 | 51 | 52 | 53 | 54 | -------------------------------------------------------------------------------- /chapter-04/sqoop-eval-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-sqoop-eval-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - Sqoop-eval action 27 | Oozie book example, ch04 - Sqoop-eval action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/sqoop-eval-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/sqoop-eval-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-sqoop-eval-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/sqoop-eval-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/sqoop-eval-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | eval 36 | --connect 37 | jdbc:mysql://mysqlhost.mycompany.com/MY_DB 38 | --username 39 | mytestsqoop 40 | --password 41 | password 42 | -e 43 | SELECT count(*) FROM test_table 44 | 45 | 46 | 47 | 48 | 49 | 50 | The example Sqoop-eval action failed! 51 | 52 | 53 | 54 | 55 | 56 | 57 | -------------------------------------------------------------------------------- /chapter-04/sqoop-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-sqoop-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - Sqoop action 27 | Oozie book example, ch04 - Sqoop action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/sqoop-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/sqoop-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-sqoop-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/sqoop-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/sqoop-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | jt.mycompany.com:8032 34 | hdfs://nn.mycompany.com:8020 35 | 36 | 37 | 38 | 39 | 40 | mapred.job.queue.name 41 | default 42 | 43 | 44 | import --connect jdbc:mysql://mysqlhost.mycompany.com/MY_DB 45 | --table test_table -username mytestsqoop -password password 46 | --target-dir /user/alti-test-01/ara/output-data/sqoop -m 1 47 | 48 | 49 | 50 | 51 | 52 | 53 | The example Sqoop action failed! 54 | 55 | 56 | 57 | 58 | 59 | 60 | -------------------------------------------------------------------------------- /chapter-04/ssh-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-ssh-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - SSH action 27 | Oozie book example, ch04 - SSH action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/ssh-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/ssh-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-ssh-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/ssh-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/ssh-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | foo@bar.com 34 | uploaddata 35 | jdbc:derby://bar.com:1527/myDB 36 | hdfs://foobar.com:8020/usr/joe/myData 37 | 38 | 39 | 40 | 41 | 42 | 43 | The example SSH action failed! 44 | 45 | 46 | 47 | 48 | 49 | 50 | -------------------------------------------------------------------------------- /chapter-04/subworkflow-wf/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 19 | 22 | 4.0.0 23 | com.oreilly.ooziebook.examples 24 | ch04-subworkflow-wf 25 | 1.0-SNAPSHOT 26 | O'Reilly Oozie book example, chapter 04 - Sub-Workflow action 27 | Oozie book example, ch04 - Sub-Workflow action 28 | jar 29 | 30 | 31 | ${artifactId} 32 | 1.2.0 33 | 34 | 35 | 36 | 37 | org.apache.hadoop 38 | hadoop-client 39 | provided 40 | ${hadoop.version} 41 | 42 | 43 | 44 | 45 | 46 | 47 | org.apache.maven.plugins 48 | maven-assembly-plugin 49 | 50 | false 51 | false 52 | example 53 | 54 | src/main/assemblies/example.xml 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /chapter-04/subworkflow-wf/src/main/assemblies/example.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | example 20 | 21 | dir 22 | 23 | false 24 | 25 | 26 | 27 | ${basedir}/src/main/xml 28 | ${deploy.dir}/app 29 | 30 | 31 | ${basedir}/src/main/data 32 | ${deploy.dir}/data 33 | 34 | 35 | ${basedir}/src/main/client 36 | / 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /chapter-04/subworkflow-wf/src/main/client/job.properties: -------------------------------------------------------------------------------- 1 | # 2 | # Licensed to the Apache Software Foundation (ASF) under one 3 | # or more contributor license agreements. See the NOTICE file 4 | # distributed with this work for additional information 5 | # regarding copyright ownership. The ASF licenses this file 6 | # to you under the Apache License, Version 2.0 (the 7 | # "License"); you may not use this file except in compliance 8 | # with the License. You may obtain a copy of the License at 9 | # 10 | # http://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | 19 | exampleDir=${nameNode}/user/${user.name}/ch04-subworkflow-wf 20 | 21 | oozie.wf.application.path=${exampleDir}/app 22 | -------------------------------------------------------------------------------- /chapter-04/subworkflow-wf/src/main/data/input/input.txt: -------------------------------------------------------------------------------- 1 | hello hello hello 2 | bye bye -------------------------------------------------------------------------------- /chapter-04/subworkflow-wf/src/main/xml/workflow.xml: -------------------------------------------------------------------------------- 1 | 18 | 19 | 20 | 21 | 22 | jobTracker 23 | 24 | 25 | nameNode 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | hdfs://nn.mycompany.com:8020/hdfs/user/joe/sub_workflow 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | The example Sub-Worflow action failed! 42 | 43 | 44 | 45 | 46 | 47 | 48 | --------------------------------------------------------------------------------