├── .github └── workflows │ ├── build.yml │ └── release.yml ├── .gitignore ├── LICENSE ├── README.md ├── build.gradle ├── gradle └── wrapper │ ├── gradle-wrapper.jar │ └── gradle-wrapper.properties ├── gradlew ├── gradlew.bat ├── src ├── main │ └── kotlin │ │ └── kscript │ │ ├── KscriptUtil.kt │ │ ├── experimental │ │ └── Incubator.kt │ │ ├── text │ │ ├── OverloadHelper.kt │ │ ├── StreamUtil.kt │ │ └── Tables.kt │ │ └── util │ │ ├── DocOpt.kt │ │ └── OneLinerContext.kt └── test │ ├── kotlin │ └── kscript │ │ ├── examples │ │ ├── AwkComparison.kt │ │ └── OneLinerExample.kt │ │ └── test │ │ ├── DocOptTest.kt │ │ ├── SupportApiTest.kt │ │ └── TestIncubator.kt │ └── resources │ ├── flights.tsv │ ├── flights.tsv.gz │ ├── flights_columns.txt │ ├── flights_head.txt │ └── some_flights.tsv └── suport_api_notes.md /.github/workflows/build.yml: -------------------------------------------------------------------------------- 1 | name: build 2 | 3 | # https://docs.github.com/en/free-pro-team@latest/actions/managing-workflow-runs/adding-a-workflow-status-badge 4 | 5 | on: 6 | push: 7 | branches: [ master ] 8 | pull_request: 9 | branches: [ master ] 10 | 11 | jobs: 12 | build: 13 | 14 | runs-on: ubuntu-latest 15 | 16 | steps: 17 | - uses: actions/checkout@v2 18 | - name: Set up JDK 11 19 | uses: actions/setup-java@v1 20 | with: 21 | java-version: 11 22 | - name: Grant execute permission for gradlew 23 | run: chmod +x gradlew 24 | - name: Build and test with Gradle 25 | # https://stackoverflow.com/questions/50104666/gradle-difference-between-test-and-check 26 | # https://stackoverflow.com/questions/50104666/gradle-difference-between-test-and-check 27 | run: ./gradlew clean check --stacktrace --info -------------------------------------------------------------------------------- /.github/workflows/release.yml: -------------------------------------------------------------------------------- 1 | # Created with https://github.com/marketplace/actions/create-a-release 2 | 3 | on: 4 | push: 5 | # Sequence of patterns matched against refs/tags 6 | # branches: 7 | # - master 8 | tags: 9 | - 'v*' # Push events to matching v*, i.e. v1.0, v20.15.10 10 | 11 | name: Create Release 12 | 13 | 14 | jobs: 15 | build: 16 | name: Create Release 17 | runs-on: ubuntu-latest 18 | steps: 19 | - name: Checkout code 20 | uses: actions/checkout@v2 21 | # https://github.community/t/accessing-commit-message-in-pull-request-event/17158/8 22 | # - name: get commit message 23 | # run: | 24 | # echo ::set-env name=commitmsg::$(git log --format=%B -n 1 ${{ github.event.after }}) 25 | # - name: show commit message 26 | # run: echo $commitmsg 27 | - name: Create Release 28 | id: create_release 29 | uses: actions/create-release@v1 30 | env: 31 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # This token is provided by Actions, you do not need to create your own token 32 | with: 33 | # https://stackoverflow.com/questions/63619329/github-action-get-commit-message 34 | # tag_name: ${{ github.event.head_commit.message }} 35 | # release_name: ${{ github.event.head_commit.message }} 36 | tag_name: ${{ github.ref }} 37 | release_name: ${{ github.ref }} 38 | body: | 39 | See [CHANGES.md](https://github.com/holgerbrandl/krangl/blob/master/CHANGES.md) for new features, bug-fixes and changes. 40 | draft: false 41 | prerelease: false -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | #gradle 2 | .gradle 3 | /build/ 4 | 5 | # IntelliJ 6 | out/ 7 | .idea 8 | *iml 9 | 10 | 11 | # sonatype credentials 12 | local.properties 13 | *.gpg -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2017 Holger Brandl 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Support API for `kscript` 2 | ========================= 3 | 4 | 5 | [ ![Download](https://img.shields.io/badge/Maven%20Central-1.2.5-orange) ](https://mvnrepository.com/artifact/com.github.holgerbrandl/kscript-support-api) [![Build Status](https://github.com/holgerbrandl/krangl/workflows/build/badge.svg)](https://github.com/holgerbrandl/kscript-support-api/actions?query=workflow%3Abuild) 6 | 7 | 8 | 9 | A support API written in Kotlin to simplify scripting with [`kscript`](https://github.com/holgerbrandl/kscript) 10 | 11 | 12 | **For documentation see [kscript manual](https://github.com/holgerbrandl/kscript#support-api)** 13 | 14 | 15 | Support 16 | ------- 17 | 18 | 19 | Feel welcome to report issues or suggestions [here](https://github.com/holgerbrandl/kscript/issues) 20 | 21 | 22 | Installation 23 | ------------ 24 | 25 | Even if the artifact is intended to be used alongside with `kscript` you can also use it without to add enhanced DocOpt support, various CLI and data streaming utilities to your project. 26 | 27 | To get started simply add it as a dependency via Jcenter: 28 | ``` 29 | compile "com.github.holgerbrandl:kscript-support-api:1.2.5" 30 | ``` 31 | 32 | 33 | License 34 | ------- 35 | 36 | [MIT](LICENSE) 37 | -------------------------------------------------------------------------------- /build.gradle: -------------------------------------------------------------------------------- 1 | //file:noinspection DifferentKotlinGradleVersion 2 | buildscript { 3 | ext.kotlin_version = '1.4.32' 4 | 5 | repositories { 6 | mavenCentral() 7 | } 8 | 9 | dependencies { 10 | classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version" 11 | } 12 | } 13 | 14 | 15 | plugins { 16 | id "java-library" 17 | // id "application" 18 | id "maven-publish" 19 | id "signing" 20 | id "io.github.gradle-nexus.publish-plugin" version "1.1.0" 21 | id "org.jetbrains.kotlin.jvm" version "1.4.32" 22 | } 23 | 24 | 25 | defaultTasks 'run' 26 | 27 | repositories { 28 | mavenCentral() 29 | // mavenLocal() 30 | } 31 | 32 | // https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plugin_and_dependency_management 33 | dependencies { 34 | compileOnly "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version" 35 | // compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version" 36 | compile "com.offbytwo:docopt:0.6.0.20150202" 37 | 38 | testCompile 'junit:junit:4.11' 39 | testCompile "io.kotlintest:kotlintest:2.0.1" 40 | } 41 | 42 | java { 43 | withJavadocJar() 44 | withSourcesJar() 45 | } 46 | 47 | group = 'com.github.holgerbrandl' 48 | version = '1.2.5' 49 | 50 | 51 | publishing { 52 | publications { 53 | mavenJava(MavenPublication) { 54 | from(components.java) 55 | 56 | pom { 57 | name = "kscript-support-api" 58 | description = 'A support API to simplify scripting with kscript' 59 | url = 'https://github.com/holgerbrandl/kscript-support-api' 60 | 61 | licenses { 62 | license { 63 | name = 'MIT' 64 | url = 'https://github.com/holgerbrandl/kscript-support-api/blob/master/LICENSE' 65 | } 66 | } 67 | 68 | 69 | scm { 70 | connection = 'scm:git:github.com/holgerbrandl/kscript-support-api.git' 71 | url = 'https://github.com/holgerbrandl/kscript-support-api.git' 72 | } 73 | 74 | 75 | developers { 76 | developer { 77 | id = 'holgerbrandl' 78 | name = 'Holger Brandl' 79 | email = 'holgerbrandl@gmail.com' 80 | } 81 | } 82 | } 83 | } 84 | } 85 | } 86 | 87 | 88 | nexusPublishing { 89 | repositories { 90 | sonatype { 91 | snapshotRepositoryUrl = uri(project.properties["sonatypeStagingProfileId"]) 92 | username = project.properties["ossrhUsername"] 93 | password = project.properties["ossrhPassword"] 94 | } 95 | } 96 | } 97 | 98 | signing { 99 | sign publishing.publications.mavenJava 100 | } -------------------------------------------------------------------------------- /gradle/wrapper/gradle-wrapper.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kscripting/kscript-support-api/01eddfb8809251024ac1fbab61779ca83463aa78/gradle/wrapper/gradle-wrapper.jar -------------------------------------------------------------------------------- /gradle/wrapper/gradle-wrapper.properties: -------------------------------------------------------------------------------- 1 | distributionBase=GRADLE_USER_HOME 2 | distributionPath=wrapper/dists 3 | zipStoreBase=GRADLE_USER_HOME 4 | zipStorePath=wrapper/dists 5 | distributionUrl=https\://services.gradle.org/distributions/gradle-6.7-all.zip 6 | -------------------------------------------------------------------------------- /gradlew: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | ############################################################################## 4 | ## 5 | ## Gradle start up script for UN*X 6 | ## 7 | ############################################################################## 8 | 9 | # Attempt to set APP_HOME 10 | # Resolve links: $0 may be a link 11 | PRG="$0" 12 | # Need this for relative symlinks. 13 | while [ -h "$PRG" ] ; do 14 | ls=`ls -ld "$PRG"` 15 | link=`expr "$ls" : '.*-> \(.*\)$'` 16 | if expr "$link" : '/.*' > /dev/null; then 17 | PRG="$link" 18 | else 19 | PRG=`dirname "$PRG"`"/$link" 20 | fi 21 | done 22 | SAVED="`pwd`" 23 | cd "`dirname \"$PRG\"`/" >/dev/null 24 | APP_HOME="`pwd -P`" 25 | cd "$SAVED" >/dev/null 26 | 27 | APP_NAME="Gradle" 28 | APP_BASE_NAME=`basename "$0"` 29 | 30 | # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. 31 | DEFAULT_JVM_OPTS="" 32 | 33 | # Use the maximum available, or set MAX_FD != -1 to use that value. 34 | MAX_FD="maximum" 35 | 36 | warn () { 37 | echo "$*" 38 | } 39 | 40 | die () { 41 | echo 42 | echo "$*" 43 | echo 44 | exit 1 45 | } 46 | 47 | # OS specific support (must be 'true' or 'false'). 48 | cygwin=false 49 | msys=false 50 | darwin=false 51 | nonstop=false 52 | case "`uname`" in 53 | CYGWIN* ) 54 | cygwin=true 55 | ;; 56 | Darwin* ) 57 | darwin=true 58 | ;; 59 | MINGW* ) 60 | msys=true 61 | ;; 62 | NONSTOP* ) 63 | nonstop=true 64 | ;; 65 | esac 66 | 67 | CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar 68 | 69 | # Determine the Java command to use to start the JVM. 70 | if [ -n "$JAVA_HOME" ] ; then 71 | if [ -x "$JAVA_HOME/jre/sh/java" ] ; then 72 | # IBM's JDK on AIX uses strange locations for the executables 73 | JAVACMD="$JAVA_HOME/jre/sh/java" 74 | else 75 | JAVACMD="$JAVA_HOME/bin/java" 76 | fi 77 | if [ ! -x "$JAVACMD" ] ; then 78 | die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME 79 | 80 | Please set the JAVA_HOME variable in your environment to match the 81 | location of your Java installation." 82 | fi 83 | else 84 | JAVACMD="java" 85 | which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 86 | 87 | Please set the JAVA_HOME variable in your environment to match the 88 | location of your Java installation." 89 | fi 90 | 91 | # Increase the maximum file descriptors if we can. 92 | if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then 93 | MAX_FD_LIMIT=`ulimit -H -n` 94 | if [ $? -eq 0 ] ; then 95 | if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then 96 | MAX_FD="$MAX_FD_LIMIT" 97 | fi 98 | ulimit -n $MAX_FD 99 | if [ $? -ne 0 ] ; then 100 | warn "Could not set maximum file descriptor limit: $MAX_FD" 101 | fi 102 | else 103 | warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT" 104 | fi 105 | fi 106 | 107 | # For Darwin, add options to specify how the application appears in the dock 108 | if $darwin; then 109 | GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" 110 | fi 111 | 112 | # For Cygwin, switch paths to Windows format before running java 113 | if $cygwin ; then 114 | APP_HOME=`cygpath --path --mixed "$APP_HOME"` 115 | CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` 116 | JAVACMD=`cygpath --unix "$JAVACMD"` 117 | 118 | # We build the pattern for arguments to be converted via cygpath 119 | ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null` 120 | SEP="" 121 | for dir in $ROOTDIRSRAW ; do 122 | ROOTDIRS="$ROOTDIRS$SEP$dir" 123 | SEP="|" 124 | done 125 | OURCYGPATTERN="(^($ROOTDIRS))" 126 | # Add a user-defined pattern to the cygpath arguments 127 | if [ "$GRADLE_CYGPATTERN" != "" ] ; then 128 | OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)" 129 | fi 130 | # Now convert the arguments - kludge to limit ourselves to /bin/sh 131 | i=0 132 | for arg in "$@" ; do 133 | CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -` 134 | CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option 135 | 136 | if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition 137 | eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"` 138 | else 139 | eval `echo args$i`="\"$arg\"" 140 | fi 141 | i=$((i+1)) 142 | done 143 | case $i in 144 | (0) set -- ;; 145 | (1) set -- "$args0" ;; 146 | (2) set -- "$args0" "$args1" ;; 147 | (3) set -- "$args0" "$args1" "$args2" ;; 148 | (4) set -- "$args0" "$args1" "$args2" "$args3" ;; 149 | (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; 150 | (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; 151 | (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; 152 | (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; 153 | (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; 154 | esac 155 | fi 156 | 157 | # Escape application args 158 | save () { 159 | for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done 160 | echo " " 161 | } 162 | APP_ARGS=$(save "$@") 163 | 164 | # Collect all arguments for the java command, following the shell quoting and substitution rules 165 | eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" 166 | 167 | # by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong 168 | if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then 169 | cd "$(dirname "$0")" 170 | fi 171 | 172 | exec "$JAVACMD" "$@" 173 | -------------------------------------------------------------------------------- /gradlew.bat: -------------------------------------------------------------------------------- 1 | @if "%DEBUG%" == "" @echo off 2 | @rem ########################################################################## 3 | @rem 4 | @rem Gradle startup script for Windows 5 | @rem 6 | @rem ########################################################################## 7 | 8 | @rem Set local scope for the variables with windows NT shell 9 | if "%OS%"=="Windows_NT" setlocal 10 | 11 | set DIRNAME=%~dp0 12 | if "%DIRNAME%" == "" set DIRNAME=. 13 | set APP_BASE_NAME=%~n0 14 | set APP_HOME=%DIRNAME% 15 | 16 | @rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. 17 | set DEFAULT_JVM_OPTS= 18 | 19 | @rem Find java.exe 20 | if defined JAVA_HOME goto findJavaFromJavaHome 21 | 22 | set JAVA_EXE=java.exe 23 | %JAVA_EXE% -version >NUL 2>&1 24 | if "%ERRORLEVEL%" == "0" goto init 25 | 26 | echo. 27 | echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 28 | echo. 29 | echo Please set the JAVA_HOME variable in your environment to match the 30 | echo location of your Java installation. 31 | 32 | goto fail 33 | 34 | :findJavaFromJavaHome 35 | set JAVA_HOME=%JAVA_HOME:"=% 36 | set JAVA_EXE=%JAVA_HOME%/bin/java.exe 37 | 38 | if exist "%JAVA_EXE%" goto init 39 | 40 | echo. 41 | echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 42 | echo. 43 | echo Please set the JAVA_HOME variable in your environment to match the 44 | echo location of your Java installation. 45 | 46 | goto fail 47 | 48 | :init 49 | @rem Get command-line arguments, handling Windows variants 50 | 51 | if not "%OS%" == "Windows_NT" goto win9xME_args 52 | 53 | :win9xME_args 54 | @rem Slurp the command line arguments. 55 | set CMD_LINE_ARGS= 56 | set _SKIP=2 57 | 58 | :win9xME_args_slurp 59 | if "x%~1" == "x" goto execute 60 | 61 | set CMD_LINE_ARGS=%* 62 | 63 | :execute 64 | @rem Setup the command line 65 | 66 | set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar 67 | 68 | @rem Execute Gradle 69 | "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS% 70 | 71 | :end 72 | @rem End local scope for the variables with windows NT shell 73 | if "%ERRORLEVEL%"=="0" goto mainEnd 74 | 75 | :fail 76 | rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of 77 | rem the _cmd.exe /c_ return code! 78 | if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 79 | exit /b 1 80 | 81 | :mainEnd 82 | if "%OS%"=="Windows_NT" endlocal 83 | 84 | :omega 85 | -------------------------------------------------------------------------------- /src/main/kotlin/kscript/KscriptUtil.kt: -------------------------------------------------------------------------------- 1 | package kscript 2 | 3 | import kotlin.system.exitProcess 4 | 5 | /** 6 | * @author Holger Brandl 7 | */ 8 | 9 | /** 10 | * Just used internally to prevent [stopIfNot] to quit the process when running in unit-test mode. 11 | * It throw an IllegalArgumentException instead. 12 | */ 13 | internal var isTestMode = false 14 | 15 | 16 | /** Similar to require but without a stacktrace which makes it more suited for CLIs. */ 17 | fun stopIfNot(value: Boolean, lazyMessage: () -> Any) { 18 | if (value) return 19 | 20 | val msg = "[ERROR] " + lazyMessage().toString() 21 | 22 | if (isTestMode) throw IllegalArgumentException(msg) 23 | 24 | System.err.println(msg) 25 | exitProcess(1) 26 | } -------------------------------------------------------------------------------- /src/main/kotlin/kscript/experimental/Incubator.kt: -------------------------------------------------------------------------------- 1 | package kscript.experimental 2 | 3 | import java.io.File 4 | 5 | /** 6 | * Experimental support methods. Those may change without further notice. 7 | * 8 | * @author Holger Brandl 9 | */ 10 | 11 | 12 | fun File.mapLines(trafo: (String) -> T) { 13 | return useLines { 14 | it.map { trafo(it) } 15 | } 16 | } 17 | 18 | fun String.processLines(trafo: (String) -> String) { 19 | split("\n").map { println(trafo(it)) } 20 | } 21 | 22 | 23 | fun processStdin(trafo: (String) -> String) { 24 | generateSequence() { readLine() }.map { 25 | println(trafo(it)) 26 | } 27 | } 28 | 29 | 30 | operator fun File.div(childName: String): File { 31 | return this.resolve(childName) 32 | } -------------------------------------------------------------------------------- /src/main/kotlin/kscript/text/OverloadHelper.kt: -------------------------------------------------------------------------------- 1 | package kscript.text 2 | 3 | /** Declare another extension on print to faciliate printing of more base split lines.*/ 4 | 5 | fun Sequence>.print(separator: String = "\t") = map { Row(it) }.print(separator) 6 | -------------------------------------------------------------------------------- /src/main/kotlin/kscript/text/StreamUtil.kt: -------------------------------------------------------------------------------- 1 | package kscript.text 2 | 3 | 4 | /** A `Sequence` iterator for standard input */ 5 | public val stdin by lazy { generateSequence() { readLine() } } 6 | 7 | fun linesFrom(file: java.io.File) = java.io.BufferedReader(java.io.FileReader(file)).lineSequence() 8 | 9 | 10 | /** 11 | * File argument processor that works similar to awk: If data is available on stdin, use it. If not expect a file argument and read from that one instead. 12 | * */ 13 | fun resolveArgFile(args: Array, position: Int = 0): Sequence { 14 | // if (stdin.iterator().hasNext()) return stdin 15 | if (System.`in`.available() > 0) return kscript.text.stdin 16 | 17 | kscript.stopIfNot(args.isNotEmpty()) { "Missing file or input input stream" } 18 | kscript.stopIfNot(args.size >= position) { "arg position ${position} exceeds number of arguments ${args.size} " } 19 | 20 | val fileArg = args[position] 21 | 22 | // stdinNames: List = listOf("-", "stdin") 23 | // if (stdinNames.contains(fileArg)) return stdin 24 | 25 | val inputFile = java.io.File(fileArg) 26 | 27 | kscript.stopIfNot(inputFile.canRead()) { "Can not read from '${fileArg}'" } 28 | 29 | // test for compression and uncompress files automatically 30 | val isCompressedInput = inputFile.name.run { endsWith(".zip") || endsWith(".gz") } 31 | 32 | val lineReader = if (isCompressedInput) { 33 | java.io.InputStreamReader(java.util.zip.GZIPInputStream(java.io.FileInputStream(inputFile))) 34 | } else { 35 | java.io.FileReader(inputFile) 36 | } 37 | 38 | 39 | // todo we don't close the buffer with this approach 40 | // BufferedReader(FileReader(inputFile )).use { return it } 41 | return java.io.BufferedReader(lineReader).lineSequence() 42 | } 43 | 44 | 45 | /** Endpoint for a kscript pipe. */ 46 | fun Sequence.print() = forEach { println(it) } 47 | 48 | /** Endpoint for a kscript pipe. */ 49 | fun Iterable.print() = forEach { println(it) } 50 | 51 | 52 | fun Iterable.trim() = map { it.trim() } 53 | fun Sequence.trim() = map { it.trim() } 54 | 55 | 56 | //https://dzone.com/articles/readingwriting-compressed-and 57 | /** Save a list of items into a file. Output can be option ally zipped and a the stringifying operation can be changed from toString to custom operation if needed. */ 58 | fun Iterable.saveAs(f: java.io.File, 59 | transform: (T) -> String = { it.toString() }, 60 | separator: Char = '\n', 61 | overwrite: Boolean = true, 62 | compress: Boolean = f.name.let { it.endsWith(".zip") || it.endsWith(".gz") }) { 63 | 64 | // ensure that file is not yet there or overwrite flag is set 65 | require(!f.isFile || overwrite) { "$f is present already. Use overwrite=true to enforce file replacement." } 66 | 67 | val p = if (!compress) java.io.PrintWriter(f) else java.io.BufferedWriter(java.io.OutputStreamWriter(java.util.zip.GZIPOutputStream(java.io.FileOutputStream(f)))) 68 | 69 | toList().forEach { p.write(transform(it) + separator) } 70 | 71 | p.close() 72 | } 73 | 74 | 75 | -------------------------------------------------------------------------------- /src/main/kotlin/kscript/text/Tables.kt: -------------------------------------------------------------------------------- 1 | package kscript.text 2 | 3 | import kscript.stopIfNot 4 | 5 | /** 6 | * Utility methods to allow for awk-like data processing using kscript. 7 | * 8 | * The general usage pattern is to start with a `Sequence` that is typically provided as `lines` by [kscript.resolveArgFile], and to end with one of the `print` extension methods provided below. 9 | * 10 | * In one-liners `lines` is implicitly added by `kscript` and the alle elements from `kscript.*` are imported. This allows for constructs such as 11 | * ``` 12 | * kscript 'lines.filter{ it.contains("foo") }.print()' some_file.txt 13 | * kscript 'lines.filter{ it.contains("foo") }.print()' 14 | * cat some_file.txt | kscript 'lines.filter{ it.contains("foo") }.print()' 15 | * 16 | * ``` 17 | * 18 | * @author Holger Brandl 19 | */ 20 | 21 | 22 | // for top-level vs member extensions see https://kotlinlang.org/docs/reference/extensions.html#scope-of-extensions 23 | 24 | 25 | /** For sake of readability we refer to a list of strings as a row here. */ 26 | //typealias Row = List 27 | 28 | class Row(val data: List) : List by data { 29 | 30 | constructor(vararg data: String) : this(data.asList()) 31 | 32 | override fun get(index: Int): String = data[index - 1] 33 | 34 | override fun toString(): String = data.joinToString("\t") 35 | } 36 | 37 | //typealias RowSequence = Sequence 38 | //class RowSequence(input: Sequence){ 39 | //} 40 | 41 | 42 | /** Splits the lines of an input stream into [Row]s. 43 | * 44 | * @param separator The used separator character which defaults to tabs. 45 | */ 46 | fun Sequence.split(separator: String = "\t"): Sequence { 47 | return this.map { Row(it.split(separator)) } 48 | } 49 | 50 | /** awk-like convenience wrapper around split->map->join->print */ 51 | fun Sequence.awk(separator: String = "\t", rule: (Row) -> String) = split(separator).map { rule(it) }.print() 52 | 53 | 54 | fun Sequence.map(vararg rules: (Row) -> String): Sequence { 55 | return map { splitLine -> Row(rules.map { it(splitLine) }) } 56 | } 57 | 58 | /** Adds a new column to a row. */ 59 | fun Sequence.add(rule: (Row) -> String): Sequence { 60 | return map { row -> Row(*row.toTypedArray(), rule(row)) } 61 | } 62 | 63 | 64 | //@Deprecated("use kscript.support.awk() instead") 65 | //fun Sequence.splitMap(vararg rules: (Row) -> String, separator: String = "\t", joinWith: String = separator) { 66 | // map { it.split(separator).let { splitLine -> rules.map { it(splitLine) } } }.print() 67 | //} 68 | 69 | 70 | fun Sequence.join(separator: String = "\t") = map { it.joinToString(separator) } 71 | 72 | /** Joins rows with the provided `separator` and print them to `stdout`. */ 73 | fun Sequence.print(separator: String = "\t") = join(separator).print() 74 | 75 | fun List.print(separator: String = "\t") = asSequence().print(separator) 76 | 77 | 78 | // 79 | // Column Select 80 | // 81 | 82 | 83 | /** Internal representations for column selection indices. Usually not use directly but rather via [with] and [without]. 84 | */ 85 | abstract class ColSelect(val indices: Array = emptyArray()) { 86 | 87 | // irrespective of selection mode (positive or negative) indices must be positive in here 88 | init { 89 | stopIfNot(indices.all { it > 0 }) { 90 | "kscript.text.* is using 1-based arrays to ease awk transition, so indices must be strictly positive" 91 | } 92 | } 93 | 94 | abstract fun and(column: Int): ColSelect 95 | abstract fun and(range: IntRange): ColSelect 96 | abstract fun process(lines: Sequence): Sequence 97 | } 98 | 99 | class PosSelect(columnIndices: Array) : ColSelect(columnIndices) { 100 | override fun and(column: Int) = PosSelect(arrayOf(*indices, column)) 101 | override fun and(range: IntRange) = PosSelect(arrayOf(*indices, *range.toList().toTypedArray())) 102 | 103 | override fun process(lines: Sequence): Sequence = lines.map { row -> Row(indices.map { row[it] }) } 104 | } 105 | 106 | class NegSelect(columnIndices: Array) : ColSelect(columnIndices) { 107 | override fun and(column: Int) = NegSelect(arrayOf(*indices, column)) 108 | override fun and(range: IntRange) = NegSelect(arrayOf(*indices, *range.toList().toTypedArray())) 109 | 110 | override fun process(lines: Sequence): Sequence = lines.map { 111 | Row(it.filterIndexed { index, _ -> !indices.contains(index+1) }) 112 | } 113 | } 114 | 115 | /** Starts building a column selection index. Both positive and negative indices are supported. */ 116 | fun with(index: Int) = PosSelect(arrayOf(index)) 117 | 118 | fun with(range: IntRange) = PosSelect(range.toList().toTypedArray()) 119 | fun without(index: Int) = NegSelect(arrayOf(index)) 120 | fun without(range: IntRange) = NegSelect(range.toList().toTypedArray()) 121 | 122 | 123 | /** 124 | * Select or remove columns by providing an index-vector. Positive selections are done with [with] and negative selections with [without]. Both methods implement a [builder][https://en.wikipedia.org/wiki/Builder_pattern] to construct more complex selectors. 125 | */ 126 | fun Sequence.select(vararg colIndices: Int): Sequence { 127 | val isPositive = colIndices.all { it > 0 } 128 | val isNegative = colIndices.all { it < 0 } 129 | 130 | stopIfNot(isPositive xor isNegative) { 131 | "Can not mix positive and negative selections" 132 | } 133 | 134 | val selector = if (isPositive) { 135 | PosSelect(arrayOf(*colIndices.toTypedArray())) 136 | } else { 137 | NegSelect(arrayOf(*colIndices.map { -it }.toTypedArray())) 138 | } 139 | 140 | return select(selector) 141 | } 142 | 143 | fun Sequence.select(columnSelector: ColSelect): Sequence { 144 | // more efficient but does not allow to change the order 145 | // return map { it.filterIndexed { index, _ -> retainColumn(columnSelector, index + 1) } } 146 | 147 | return columnSelector.process(this) 148 | } 149 | 150 | 151 | // http://www.thegeekstuff.com/2010/01/8-powerful-awk-built-in-variables-fs-ofs-rs-ors-nr-nf-filename-fnr/?ref=binfind.com/web 152 | //NF Example: Number of Fields in a record 153 | //val LAST = Integer.MIN_VALUE 154 | 155 | // todo add krangl ColNames interface here 156 | 157 | -------------------------------------------------------------------------------- /src/main/kotlin/kscript/util/DocOpt.kt: -------------------------------------------------------------------------------- 1 | package kscript.util 2 | 3 | import org.docopt.Docopt 4 | import java.io.File 5 | 6 | /** 7 | * Helpers to build CLIs with kscript and docopt 8 | * 9 | * @author Holger Brandl 10 | */ 11 | 12 | /** Simple Kotlin facade for org.docopt.Docopt.Docopt(java.lang.String) .*/ 13 | class DocOpt(args: Array, val usage: String) { 14 | 15 | val parsedArgs = Docopt(usage).parse(args.toList()) 16 | 17 | private val myDO by lazy { 18 | parsedArgs.map { 19 | it.key.removePrefix("--").replace("[<>]".toRegex(), "") to it.value 20 | }.toMap() 21 | } 22 | 23 | fun getString(key: String) = myDO[key]!!.toString() 24 | fun getStrings(key: String) = (myDO[key]!! as List<*>).map { it as String } 25 | 26 | fun getFile(key: String) = File(getString(key)) 27 | fun getFiles(key: String) = getStrings(key).map { File(it) } 28 | 29 | 30 | fun getInt(key: String) = myDO[key]!!.toString().toInt() 31 | 32 | fun getNumber(key: String) = myDO[key]!!.toString().toFloat() 33 | 34 | fun getBoolean(key: String) = myDO[key]!!.toString().toBoolean() 35 | 36 | override fun toString(): String { 37 | return parsedArgs.toString() 38 | } 39 | } -------------------------------------------------------------------------------- /src/main/kotlin/kscript/util/OneLinerContext.kt: -------------------------------------------------------------------------------- 1 | package kscript.util 2 | 3 | import kscript.text.resolveArgFile 4 | 5 | /** 6 | * @author Holger Brandl 7 | */ 8 | 9 | abstract class OneLinerContext(args: Array) { 10 | 11 | val arg by lazy { resolveArgFile(args) } 12 | // val stdin by lazy { kscript.support.getStdin } 13 | 14 | init { 15 | apply(arg) 16 | } 17 | 18 | abstract fun apply(lines: Sequence) 19 | } 20 | 21 | -------------------------------------------------------------------------------- /src/test/kotlin/kscript/examples/AwkComparison.kt: -------------------------------------------------------------------------------- 1 | package kscript.examples 2 | 3 | import kscript.stopIfNot 4 | import kscript.util.OneLinerContext 5 | import kscript.text.* 6 | 7 | /** 8 | * @author Holger Brandl 9 | */ 10 | 11 | val args = arrayOf("flights.txt") 12 | 13 | object AwkExample : OneLinerContext(args) { 14 | override fun apply(lines: Sequence) { 15 | lines.split().map({ it[1] }, { it[2] }).print() 16 | 17 | lines.split().filter { it[3].matches("UA".toRegex()) }.print() 18 | 19 | lines.drop(1).split().filter { it[3].matches("UA".toRegex()) }.print() 20 | 21 | 22 | // positive selection 23 | lines.split().select(with(1..3).and(3)).print() 24 | // negative selection 25 | lines.split().select(without(7).and(3..4)).print() 26 | 27 | lines.awk { it[1] + it[2] } 28 | 29 | lines.awk { it[1] + it[2] } 30 | lines.awk { it[3] } 31 | 32 | // http://stackoverflow.com/questions/15361632/delete-a-column-with-awk-or-sed 33 | // lines.split().select((-3).print() 34 | 35 | 36 | 37 | // http@ //tuxgraphics.org/~guido/scripts/awk-one-liner.html 38 | // Print the next two (i=2) lines after the line matching regexp: 39 | // awk '/regexp/{i=2;next;}{if(i){i--; print;}}' file.txt 40 | 41 | 42 | fun List.sliding(windowSize: Int): List> { 43 | return this.dropLast(windowSize - 1).mapIndexed { i, s -> this.subList(i, i + windowSize) } 44 | } 45 | 46 | val regex = "a[bc]+".toRegex() 47 | 48 | resolveArgFile(args). 49 | toList().sliding(4). 50 | filter { it[0].matches(regex) }. 51 | flatten().print() 52 | 53 | 54 | 55 | // number lines (from http://tuxgraphics.org/~guido/scripts/awk-one-liner.html) 56 | // awk '{print FNR "\t" $0}' 57 | lines.mapIndexed { num, line -> num.toString() + " " + line }.print() 58 | 59 | // Remove duplicate consecutive lines (uniq): 60 | // awk 'a !~ $0{print}; {a=$0}' 61 | 62 | 63 | // Delete trailing white space (spaces, tabs) 64 | // awk '{sub(/[ \t]*$/, "");print}' file.txt 65 | lines.map { it.trim() }.print() 66 | 67 | // Count lines (wc -l): 68 | // awk 'END{print NR}' 69 | println(lines.fold(0) { cur, _ -> cur + 1 }) 70 | println(lines.mapIndexed { i, _ -> i }.last()) 71 | // don't 72 | println(lines.toList().size) 73 | 74 | 75 | 76 | // Print the lines from a file starting at the line matching "start" until the line matching "stop": 77 | // awk '/start/,/stop/' file.txt 78 | 79 | lines.dropWhile { it.startsWith("foo") }.takeWhile { it.startsWith("bar") }.print() 80 | 81 | 82 | // Print the last field in each line: 83 | // awk -F: '{ print $NF }' file.txt 84 | lines.split(":").map { it[it.size - 1] }.print() 85 | 86 | 87 | 88 | // Prints Record(line) number, and number of fields in that record 89 | // awk '{print NR,"->",NF}' file.txt 90 | lines.split().mapIndexed { index, row -> index.toString() + " -> " + row.size }.print() 91 | lines.split().mapIndexed { index, row -> index.toString() + " -> " + row.size }.print() 92 | 93 | 94 | val arg by lazy { resolveArgFile(args) } 95 | arg.filter { true }.print() 96 | 97 | lines.split().select(with(0).and(1)).print() 98 | lines.map { it.replace("NA\"; exonic_part_number \"", "NA\"; exonic_part_number \"i") }.print() 99 | 100 | lines.split().mapIndexed { index, row -> "$index -> " + row.size }.print() 101 | } 102 | } 103 | //file:///Users/brandl/Desktop/awk_cheatsheets.pdf 104 | 105 | 106 | -------------------------------------------------------------------------------- /src/test/kotlin/kscript/examples/OneLinerExample.kt: -------------------------------------------------------------------------------- 1 | package kscript.examples 2 | 3 | import kscript.util.OneLinerContext 4 | import kscript.text.add 5 | import kscript.text.print 6 | import kscript.text.split 7 | 8 | /** 9 | * One-liner kscript example. To ease development simply extend [OneLinerContext] as shown, which will provide 10 | * the same context as `kscript` when running in single line mode. 11 | * 12 | * @author Holger Brandl 13 | */ 14 | 15 | fun main(args: Array) { 16 | 17 | object : OneLinerContext(args) { 18 | 19 | override fun apply(lines: Sequence) { 20 | lines.split().drop(1).filter { it[9] == "UA" }.add { it[3] + ":" + it[3] }.print() 21 | // kscript 'lines.split().drop(1).filter { it[9] == "UA" }.add { it[3] + ":" + it[3] }.print()' 22 | } 23 | } 24 | } -------------------------------------------------------------------------------- /src/test/kotlin/kscript/test/DocOptTest.kt: -------------------------------------------------------------------------------- 1 | package kscript.test 2 | 3 | import io.kotlintest.matchers.shouldBe 4 | import io.kotlintest.specs.StringSpec 5 | import kscript.util.DocOpt 6 | import java.io.File 7 | 8 | /** 9 | * @author Holger Brandl 10 | */ 11 | 12 | 13 | class DocOptTest : StringSpec() { init { 14 | 15 | 16 | val usage = """ 17 | Use star to align fastq files against a genome 18 | Usage: star_align.kts [options] ... 19 | 20 | Options: 21 | --gtf Custom gtf file instead of igenome bundled copy 22 | --pc-only Use protein coding genes only for mapping and quantification 23 | -n --num-fragments Fragment count used for processing [default: 5] 24 | """ 25 | 26 | val args = "-n 7 --pc-only --gtf my.gtf genome.fasta a.fastq b.fastq".split(" ").toTypedArray() 27 | 28 | 29 | val docopt = DocOpt(args, usage) 30 | 31 | 32 | // optional arg 33 | docopt.getString("gtf") shouldBe "my.gtf" 34 | docopt.getFile("gtf") shouldBe File("my.gtf") 35 | 36 | // mandatory arg 37 | docopt.getString("igenome") shouldBe "genome.fasta" 38 | docopt.getFile("igenome") shouldBe File("genome.fasta") 39 | 40 | docopt.getStrings("fastq_files") shouldBe listOf("a.fastq", "b.fastq") 41 | docopt.getFiles("fastq_files") shouldBe listOf(File("a.fastq"), File("b.fastq")) 42 | 43 | 44 | docopt.getInt("num-fragments") shouldBe 7 45 | docopt.getNumber("num-fragments").toString() shouldBe "7.0" 46 | docopt.getBoolean("pc-only") shouldBe true 47 | } 48 | } -------------------------------------------------------------------------------- /src/test/kotlin/kscript/test/SupportApiTest.kt: -------------------------------------------------------------------------------- 1 | package kscript.test 2 | 3 | import io.kotlintest.matchers.* 4 | import kscript.text.resolveArgFile 5 | import kscript.text.select 6 | import kscript.text.split 7 | import kscript.text.with 8 | import org.junit.Test 9 | 10 | /** 11 | * @author Holger Brandl 12 | */ 13 | 14 | 15 | fun someFlights() = resolveArgFile(arrayOf("src/test/resources/some_flights.tsv")) 16 | 17 | fun flightsZipped() = resolveArgFile(arrayOf("src/test/resources/flights.tsv.gz")) 18 | fun flights() = resolveArgFile(arrayOf("src/test/resources/flights.txt")) 19 | 20 | 21 | class SupportApiTest { 22 | 23 | init { 24 | kscript.isTestMode = true 25 | } 26 | 27 | 28 | @Test 29 | fun `extract field with column filter`() { 30 | someFlights().split(). 31 | filter { it[12] == "N14228" }. 32 | map { it[13] }. 33 | toList(). 34 | apply { 35 | size shouldEqual 1 36 | first() shouldEqual "EWR" 37 | } 38 | } 39 | 40 | 41 | @Test 42 | fun `allow to select column`() { 43 | someFlights().split() 44 | .select(with(3).and(11..13).and(1)) 45 | .first().data shouldBe listOf("day", "flight", "tailnum", "origin", "year") 46 | } 47 | 48 | 49 | @Test 50 | fun `is should perform a negative selection`() { 51 | someFlights().split() 52 | .select(1, 2, 3) 53 | .select(-2) 54 | .first().data shouldBe listOf("year", "day") 55 | } 56 | 57 | 58 | @Test 59 | fun `rejeced mixed select`() { 60 | shouldThrow { 61 | someFlights().split().select(1, -2) 62 | }.message shouldBe "[ERROR] Can not mix positive and negative selections" 63 | } 64 | 65 | 66 | @Test 67 | fun `compressed lines should be unzipped on the fly`() { 68 | resolveArgFile(arrayOf("src/test/resources/flights.tsv.gz")). 69 | drop(1).first() should startWith("2013") 70 | } 71 | } -------------------------------------------------------------------------------- /src/test/kotlin/kscript/test/TestIncubator.kt: -------------------------------------------------------------------------------- 1 | package kscript.test 2 | 3 | import kscript.text.* 4 | 5 | /** 6 | * @author Holger Brandl 7 | */ 8 | 9 | 10 | // just used for testing and development 11 | fun linesFrom(vararg lines: String) = lines.asSequence() 12 | 13 | 14 | fun main(args: Array) { 15 | someFlights().split().map { listOf(it[1], it[2], "F11-" + it[7]) }.print() 16 | someFlights().split().map { Row(it[1], it[2], "F11-" + it[7]) }.print() 17 | // kscript 'lines.split().map { listOf(it[1], it[2], "F11-"+ it[7]) }.print()' some_flights.tsv 18 | // kscript 'lines.split().map { Row(it[1], it[2], "F11-"+ it[7]) }.print()' some_flights.tsv 19 | 20 | someFlights().split() 21 | .select(with(3).and(11..13).and(1)) 22 | .first().print() //shouldBe listOf("day", "flight", "tailnum", "origin", "year") 23 | 24 | // remove header 25 | someFlights().drop(1).take(5) 26 | 27 | 28 | // positive selection 29 | System.out.println("positive selection") 30 | // flightsHead().split().select(with(1..3).and(3)).print() 31 | someFlights().split().select(1, 10, 12).print() 32 | someFlights().split().select(10, 1, 12).print() 33 | 34 | //create column 35 | //create column 36 | someFlights().split().map { listOf(it[1], it[2], "F11-" + it[7]) }.print() 37 | 38 | // negative selection 39 | System.out.println("negative selection") 40 | someFlights().split().select(without(7).and(3..4)).print() 41 | } -------------------------------------------------------------------------------- /src/test/resources/flights.tsv.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kscripting/kscript-support-api/01eddfb8809251024ac1fbab61779ca83463aa78/src/test/resources/flights.tsv.gz -------------------------------------------------------------------------------- /src/test/resources/flights_columns.txt: -------------------------------------------------------------------------------- 1 | $ year 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013... 2 | $ month 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1... 3 | $ day 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1... 4 | $ dep_time 517, 533, 542, 544, 554, 554, 555, 557, 557, 558, 55... 5 | $ sched_dep_time 515, 529, 540, 545, 600, 558, 600, 600, 600, 600, 60... 6 | $ dep_delay 2, 4, 2, -1, -6, -4, -5, -3, -3, -2, -2, -2, -2, -2,... 7 | $ arr_time 830, 850, 923, 1004, 812, 740, 913, 709, 838, 753, 8... 8 | $ sched_arr_time 819, 830, 850, 1022, 837, 728, 854, 723, 846, 745, 8... 9 | $ arr_delay 11, 20, 33, -18, -25, 12, 19, -14, -8, 8, -2, -3, 7,... 10 | $ carrier "UA", "UA", "AA", "B6", "DL", "UA", "B6", "EV", "B6"... 11 | $ flight 1545, 1714, 1141, 725, 461, 1696, 507, 5708, 79, 301... 12 | $ tailnum "N14228", "N24211", "N619AA", "N804JB", "N668DN", "N... 13 | $ origin "EWR", "LGA", "JFK", "JFK", "LGA", "EWR", "EWR", "LG... 14 | $ dest "IAH", "IAH", "MIA", "BQN", "ATL", "ORD", "FLL", "IA... 15 | $ air_time 227, 227, 160, 183, 116, 150, 158, 53, 140, 138, 149... 16 | $ distance 1400, 1416, 1089, 1576, 762, 719, 1065, 229, 944, 73... 17 | $ hour 5, 5, 5, 5, 6, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5, 6, 6... 18 | $ minute 15, 29, 40, 45, 0, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0, 59... 19 | $ time_hour 2013-01-01 05:00:00, 2013-01-01 05:00:00, 2013-01-0.. -------------------------------------------------------------------------------- /src/test/resources/flights_head.txt: -------------------------------------------------------------------------------- 1 | year month day dep_time dep_delay arr_time arr_delay carrier tailnum flight origin dest air_time distance hour minute 2 | 2013 1 1 517 2 830 11 UA N14228 1545 EWR IAH 227 1400 5 17 3 | 2013 1 1 533 4 850 20 UA N24211 1714 LGA IAH 227 1416 5 33 4 | 2013 1 1 542 2 923 33 AA N619AA 1141 JFK MIA 160 1089 5 42 5 | 2013 1 1 544 -1 1004 -18 B6 N804JB 725 JFK BQN 183 1576 5 44 6 | -------------------------------------------------------------------------------- /src/test/resources/some_flights.tsv: -------------------------------------------------------------------------------- 1 | year month day dep_time sched_dep_time dep_delay arr_time sched_arr_time arr_delay carrier flight tailnum origin dest air_time distance hour minute time_hour 2 | 2013 1 1 517 515 2 830 819 11 UA 1545 N14228 EWR IAH 227 1400 5 15 2013-01-01T05:00:00Z 3 | 2013 1 1 533 529 4 850 830 20 UA 1714 N24211 LGA IAH 227 1416 5 29 2013-01-01T05:00:00Z 4 | 2013 1 1 542 540 2 923 850 33 AA 1141 N619AA JFK MIA 160 1089 5 40 2013-01-01T05:00:00Z 5 | 2013 1 1 544 545 -1 1004 1022 -18 B6 725 N804JB JFK BQN 183 1576 5 45 2013-01-01T05:00:00Z 6 | -------------------------------------------------------------------------------- /suport_api_notes.md: -------------------------------------------------------------------------------- 1 | Developer Info 2 | -------------- 3 | 4 | 5 | ```bash 6 | #cd /Users/brandl/projects/kotlin/kscript-support-api 7 | ## to install to local maven repo 8 | gradle install 9 | 10 | ## to run the tests 11 | gradle test 12 | 13 | ``` 14 | 15 | Release Checklist 16 | ----------------- 17 | 18 | 1. Update version in 19 | * `build.gradle` 20 | * `kscript and` 21 | * `kscript-support-api/README` 22 | * `kscript/README` 23 | 24 | 25 | 2. Do the central release 26 | ```bash 27 | # cd /c/brandl_data/projects/misc/kscript-support-api 28 | 29 | ./gradlew install 30 | 31 | ./gradlew publishToMavenLocal 32 | 33 | #./gradlew publishToSonatype closeSonatypeStagingRepository 34 | ./gradlew publishToSonatype closeAndReleaseSonatypeStagingRepository 35 | ``` 36 | 37 | 3. Do the github source release 38 | 39 | ```bash 40 | 41 | # make sure that are no pending chanes 42 | #(git diff --exit-code && git tag v${kscript_version}) || echo "could not tag current branch" 43 | git diff --exit-code || echo "There are uncommitted changes" 44 | 45 | 46 | git tag "1.2.5" 47 | 48 | git push origin 49 | git push origin --tags 50 | ``` 51 | 52 | 53 | Links 54 | ----- 55 | 56 | 57 | http://stackoverflow.com/questions/38021360/kotlin-object-vs-companion-object-vs-package-scoped-methods 58 | 59 | --------------------------------------------------------------------------------