├── tests ├── napiclient │ ├── requirements.txt │ └── bin │ │ ├── prepare_python.sh │ │ ├── update_installation.sh │ │ └── prepare_scpmocker.pl ├── integration_tests │ ├── __init__.py │ ├── bin │ │ └── test_wrapper.sh │ ├── napi │ │ ├── cover.py │ │ ├── sandbox.py │ │ ├── __init__.py │ │ ├── subtitles.py │ │ ├── output.py │ │ ├── mock.py │ │ ├── movie_details.py │ │ ├── scpmocker.py │ │ ├── runner.py │ │ ├── testcase.py │ │ ├── blob.py │ │ ├── fs.py │ │ ├── assets.py │ │ └── xml_result.py │ ├── test_help.py │ ├── test_charset.py │ └── test_languages.py ├── napiserver │ ├── requirements.txt │ └── bin │ │ └── prepare_pretenders.sh ├── run_all_tests.sh ├── napitester │ └── bin │ │ ├── prepare_kcov.pl │ │ ├── prepare_scpmocker.pl │ │ ├── prepare_shells.pl │ │ └── prepare_assets.pl ├── Dockerfile-napiserver ├── Dockerfile-napiclient ├── common │ └── lib │ │ └── perl5 │ │ ├── GithubInstaller.pm │ │ └── NetInstall.pm ├── docker-compose.yml ├── run_integration_tests.sh ├── unit_tests │ ├── fake │ │ ├── libnapi_version_fake.sh │ │ ├── libnapi_fs_fake.sh │ │ └── libnapi_logging_fake.sh │ ├── libnapi_argv_test.sh │ ├── mock │ │ └── scpmocker.sh │ ├── libnapi_sysconf_test.sh │ ├── libnapi_xml_test.sh │ ├── libnapi_language_test.sh │ ├── libnapi_subs_test.sh │ ├── libnapi_http_test.sh │ └── libnapi_logging_test.sh ├── run_unit_tests.sh ├── Dockerfile-napitester └── README.md ├── .gitattributes ├── .agignore ├── .gitignore ├── BUGS.md ├── AUTHORS.md ├── Dockerfile ├── COLABORATION.md ├── .gitlab-ci.yml ├── CMakeLists.txt ├── libs ├── libnapi_version.sh.in ├── libnapi_retvals.sh ├── libnapi_hooks.sh ├── libnapi_constants.sh ├── libnapi_xml.sh ├── libnapi_sysconf.sh ├── libnapi_argv.sh ├── libnapi_http.sh ├── libnapi_language.sh ├── libnapi_wrappers.sh ├── libnapi_assoc.sh ├── libnapi_subs.sh └── libnapi_tools.sh ├── INSTALL.md ├── test_tools.sh ├── CodingRules.md ├── actions ├── libnapi_subtitles.sh ├── libnapi_search.sh └── libnapi_download.sh ├── CHANGELOG.md └── README.md /tests/napiclient/requirements.txt: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tests/integration_tests/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | * text=auto 2 | *.sh text eol=lf 3 | -------------------------------------------------------------------------------- /.agignore: -------------------------------------------------------------------------------- 1 | .vagrant 2 | tests/coverage 3 | *.pyc 4 | *.xml 5 | *.avi 6 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .vagrant 2 | tests/coverage 3 | *.pyc 4 | *.xml 5 | *.avi 6 | -------------------------------------------------------------------------------- /tests/napiserver/requirements.txt: -------------------------------------------------------------------------------- 1 | bottle==0.12.13 2 | pretenders==1.4.3 3 | virtualenv==15.1.0 4 | -------------------------------------------------------------------------------- /tests/napiclient/bin/prepare_python.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | pip install -r ./napiclient/requirements.txt 3 | -------------------------------------------------------------------------------- /tests/napiserver/bin/prepare_pretenders.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | pip install -r ./napiserver/requirements.txt 3 | -------------------------------------------------------------------------------- /tests/napiclient/bin/update_installation.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | cd "$(mktemp -d)" || { 4 | echo "Unable to create build directory" 5 | exit 1 6 | } 7 | 8 | cmake /mnt 9 | make && make install 10 | -------------------------------------------------------------------------------- /tests/integration_tests/bin/test_wrapper.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | declare -r LOGFILE="$1" 4 | declare -r SHELL="$2" 5 | 6 | exec 4<> "$LOGFILE" 7 | shift 2 8 | 9 | date >&4 10 | echo "cmdline: $*" >&4 11 | 12 | export BASH_XTRACEFD=4 13 | exec "${SHELL}" -x "$@" 14 | -------------------------------------------------------------------------------- /tests/run_all_tests.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | # run all tests using different implementations of awk 6 | ./run_unit_tests.sh 7 | ./run_unit_tests.sh -a /usr/bin/mawk 8 | ./run_unit_tests.sh -a /usr/bin/original-awk 9 | 10 | # run all integration tests 11 | ./run_integration_tests.sh 12 | -------------------------------------------------------------------------------- /BUGS.md: -------------------------------------------------------------------------------- 1 | If you have any proposition regarding this script or You wish to report a bug, Please, do so. Use the github Issue tracker available here: 2 | 3 | https://github.com/dagon666/napi/issues 4 | 5 | Or contact the author directly. Your contribution will help to make this tool better and more suited to Your needs. 6 | -------------------------------------------------------------------------------- /tests/napitester/bin/prepare_kcov.pl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | use strict; 4 | use warnings; 5 | $|++; 6 | 7 | use lib qw(./common/lib/perl5); 8 | use GithubInstaller; 9 | use NetInstall; 10 | 11 | GithubInstaller::preparePkg("SimonKagstrom", 12 | "kcov", 13 | "33", 14 | \&NetInstall::cmakeInstall 15 | ); 16 | 17 | __END__ 18 | 19 | -------------------------------------------------------------------------------- /tests/napiclient/bin/prepare_scpmocker.pl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | use strict; 4 | use warnings; 5 | $|++; 6 | 7 | use lib qw(./common/lib/perl5); 8 | use GithubInstaller; 9 | use NetInstall; 10 | 11 | GithubInstaller::preparePkg("dagon666", 12 | "scpmocker", 13 | "0.2", 14 | \&NetInstall::pythonInstall 15 | ); 16 | 17 | __END__ 18 | -------------------------------------------------------------------------------- /tests/napitester/bin/prepare_scpmocker.pl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | use strict; 4 | use warnings; 5 | $|++; 6 | 7 | use lib qw(./common/lib/perl5); 8 | use GithubInstaller; 9 | use NetInstall; 10 | 11 | GithubInstaller::preparePkg("dagon666", 12 | "scpmocker", 13 | "0.2", 14 | \&NetInstall::pythonInstall 15 | ); 16 | 17 | __END__ 18 | -------------------------------------------------------------------------------- /AUTHORS.md: -------------------------------------------------------------------------------- 1 | [Tomasz Wisniewski aka DAGON](tomasz.wisni3wski@gmail.com) 2 | 3 | Contributions 4 | ============= 5 | 6 | - [Michal Gorny](mgorny@gentoo.org) 7 | - [Piotr Walesiuk]() 8 | - [Mateusz "emfor"](https://github.com/emfor) 9 | - [Maciej Lopacinki](https://github.com/hamsterready) 10 | - [lukjod](https://github.com/lukjod) - major test contribution 11 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/cover.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python2 2 | 3 | from . import blob 4 | 5 | class Cover(blob.Blob): 6 | """ 7 | Abstraction around a cover delivered in XML 8 | Cover is just a base64 encoded jpg file 9 | """ 10 | 11 | def __init__(self, asset, data): 12 | super(Cover, self).__init__(asset['md5'], data) 13 | self.asset = asset 14 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/sandbox.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import os 4 | import shutil 5 | import tempfile 6 | 7 | class Sandbox(object): 8 | def __init__(self): 9 | self.create() 10 | 11 | def create(self): 12 | self.path = tempfile.mkdtemp() 13 | 14 | def destroy(self): 15 | if os.path.exists(self.path): 16 | shutil.rmtree(self.path) 17 | 18 | def __enter__(self): 19 | return self 20 | 21 | def __exit__(self, *args): 22 | self.destroy() 23 | -------------------------------------------------------------------------------- /tests/Dockerfile-napiserver: -------------------------------------------------------------------------------- 1 | FROM python:2.7 2 | 3 | ENV NAPISERVER_HOME /home/napiserver 4 | ENV NAPISERVER_OPT /opt/napiserver 5 | ENV NAPISERVER_SHARE $NAPISERVER_OPT/share 6 | ENV NAPISERVER_PORT 8000 7 | ENV INSTALL_DIR /tmp/install 8 | 9 | EXPOSE $NAPISERVER_PORT 10 | 11 | # napiserver specific 12 | ADD napiserver $INSTALL_DIR/napiserver 13 | WORKDIR $INSTALL_DIR 14 | RUN ./napiserver/bin/prepare_pretenders.sh 15 | 16 | ENTRYPOINT ["python", "-m", "pretenders.server.server", "--host", "0.0.0.0"] 17 | CMD ["--port", "8000"] 18 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM debian 2 | 3 | ENV INSTALL_DIR /tmp/napi 4 | ENV NAPI_HOME /home/napi 5 | 6 | RUN apt-get update -y 7 | RUN apt-get install -y \ 8 | make \ 9 | cmake 10 | 11 | RUN apt-get install -y \ 12 | libav-tools \ 13 | mediainfo 14 | 15 | RUN apt-get install -y \ 16 | wget \ 17 | p7zip-full \ 18 | gawk 19 | 20 | RUN useradd -m -U napi -d $NAPI_HOME 21 | RUN mkdir -p $INSTALL_DIR 22 | WORKDIR $INSTALL_DIR 23 | 24 | ADD . $INSTALL_DIR 25 | RUN mkdir -p build && \ 26 | cd build && \ 27 | cmake .. && \ 28 | make && \ 29 | make install 30 | 31 | USER napi 32 | ENTRYPOINT ["napi.sh"] 33 | -------------------------------------------------------------------------------- /tests/Dockerfile-napiclient: -------------------------------------------------------------------------------- 1 | FROM napi 2 | 3 | ENV NAPICLIENT_OPT /opt/napi 4 | ENV NAPICLIENT_TESTDATA $NAPICLIENT_OPT/testdata 5 | ENV NAPICLIENT_SHELLS $NAPICLIENT_OPT/bash 6 | 7 | USER root 8 | RUN apt-get update -y 9 | RUN apt-get install -y \ 10 | libarchive-extract-perl \ 11 | libwww-perl \ 12 | python-pip \ 13 | python-setuptools 14 | 15 | ADD common $INSTALL_DIR/common 16 | ADD napiclient $INSTALL_DIR/napiclient 17 | ADD napiserver $INSTALL_DIR/napiserver 18 | 19 | WORKDIR $INSTALL_DIR 20 | RUN ./napiserver/bin/prepare_pretenders.sh 21 | RUN ./napiclient/bin/prepare_scpmocker.pl 22 | RUN ./napiclient/bin/prepare_python.sh 23 | 24 | # go back to non-root user 25 | USER napi 26 | ENTRYPOINT [] 27 | CMD [] 28 | -------------------------------------------------------------------------------- /tests/common/lib/perl5/GithubInstaller.pm: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | use strict; 4 | use warnings; 5 | 6 | use Exporter(); 7 | use Carp; 8 | 9 | use NetInstall; 10 | 11 | package GithubInstaller; 12 | 13 | sub preparePkg { 14 | my $user = shift; 15 | my $pkg = shift; 16 | my $version = shift; 17 | my $installCmd = shift; 18 | 19 | my $upId = "${user}/${pkg}"; 20 | my $workDir = File::Temp::tempdir(CLEANUP => 1); 21 | my $url = "https://github.com/${upId}/archive/v${version}.tar.gz"; 22 | my $tgzPath = $workDir . "/v${version}.tgz"; 23 | my $srcPath = $workDir . "/${pkg}-${version}"; 24 | 25 | NetInstall::prepareTgz($url, $workDir, 26 | $tgzPath, $srcPath, 27 | "", $installCmd); 28 | } 29 | 30 | 1; 31 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import logging 4 | import sys 5 | import os 6 | 7 | def prepareLogging(logLevel): 8 | formatter = logging.Formatter('%(asctime)s %(message)s') 9 | 10 | handler = (logging.StreamHandler(sys.stderr)) 11 | handler.setFormatter(formatter) 12 | handler.setLevel(logLevel) 13 | 14 | logger = logging.getLogger() 15 | logger.setLevel(logLevel) 16 | logger.addHandler(handler) 17 | 18 | thresholdIndex = os.environ.get('NAPI_INTEGRATION_TESTS_LOGLEVEL', 0) 19 | threshold = logging.DEBUG 20 | 21 | try: 22 | thresholds = [ logging.INFO, logging.DEBUG ] 23 | mappedThreshold = thresholds[int(thresholdIndex)] 24 | threshold = mappedThreshold 25 | except IndexError, ValueError: 26 | pass 27 | 28 | prepareLogging(threshold) 29 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/subtitles.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python2 2 | 3 | from . import blob 4 | 5 | class Subtitles(blob.Blob): 6 | """ 7 | Abstraction around a subtitles file 8 | """ 9 | def __init__(self, asset, data): 10 | super(Subtitles, self).__init__(asset['md5'], data) 11 | self.asset = asset 12 | 13 | class CompressedSubtitles(blob.CompressedBlob): 14 | """ 15 | Abstraction around subtitles stored in the 7z archive 16 | """ 17 | 18 | PASSWORD = 'iBlm8NTigvru0Jr0' 19 | 20 | def __init__(self, asset, data): 21 | self.asset = asset 22 | 23 | # generate the subtitles file name 24 | try: 25 | subsName = asset['md5'] + '.txt' 26 | except KeyError: 27 | subsName = 'unknown.txt' 28 | 29 | super(CompressedSubtitles, self).__init__( 30 | asset['md5'], 31 | data, 32 | subsName, 33 | self.PASSWORD) 34 | -------------------------------------------------------------------------------- /COLABORATION.md: -------------------------------------------------------------------------------- 1 | **Bashnapi** is an open project. I accept the pull requests gladly. Please feel 2 | free to contact me if you would like to collaborate or post a patch for this 3 | project. 4 | 5 | Rules of thumb: 6 | =============== 7 | 8 | 0. Most importantly, before writing the code, read CodingRules.md. 9 | 10 | 1. Always fork the *dev* branch (your changes before merged to *master* will be 11 | merged there). *dev* represents current development state of the project and 12 | contains the newest code. 13 | 14 | 2. Before posting a patch **TEST** it. Perform some manual verification first 15 | and use bashnapi test environment to verify if your delivery didn't brake 16 | anything else. 17 | 18 | 3. Write tests for your code (refer to test suite's README.md under tests/ 19 | subdirectory for more details). 20 | 21 | 4. Don't duplicate the code. Bash itself is not very well suited for large 22 | projects and this one starts to big (for a shell script). Cross check the 23 | sources before adding new functions. 24 | -------------------------------------------------------------------------------- /.gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | image: docker:1.11 2 | 3 | services: 4 | - docker:dind 5 | 6 | stages: 7 | - build 8 | - test 9 | - deploy 10 | - pages 11 | 12 | before_script: 13 | - docker version 14 | 15 | build: 16 | stage: build 17 | script: 18 | - docker build -t napi . 19 | 20 | test: 21 | stage: test 22 | script: 23 | - apk add --no-cache py-pip 24 | - apk add --no-cache bash 25 | - apk add --no-cache jq 26 | - pip install docker-compose 27 | - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN registry.gitlab.com 28 | - docker pull registry.gitlab.com/hesperos/napi/napitester 29 | - docker tag registry.gitlab.com/hesperos/napi/napitester napitester 30 | - cd tests && ./run_unit_tests.sh 31 | artifacts: 32 | paths: 33 | - tests/coverage/ 34 | 35 | deployDocker: 36 | stage: deploy 37 | script: 38 | - echo "no deployment support yet!" 39 | 40 | pages: 41 | stage: deploy 42 | script: 43 | - mv tests/coverage/ public/ 44 | artifacts: 45 | paths: 46 | - public 47 | expire_in: 30 days 48 | only: 49 | - master 50 | -------------------------------------------------------------------------------- /tests/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '2' 2 | 3 | services: 4 | napiserver: 5 | image: registry.gitlab.com/hesperos/napi/napiserver:latest 6 | build: 7 | context: . 8 | dockerfile: Dockerfile-napiserver 9 | 10 | container_name: tests_napiserver 11 | 12 | volumes: 13 | - opt-napi:/opt/napi 14 | 15 | napitester: 16 | image: registry.gitlab.com/hesperos/napi/napitester:latest 17 | build: 18 | context: . 19 | dockerfile: Dockerfile-napitester 20 | 21 | container_name: tests_napitester 22 | 23 | volumes: 24 | - opt-napi:/opt/napi 25 | - ..:/mnt 26 | 27 | stdin_open: true 28 | working_dir: /mnt/tests/unit_tests 29 | 30 | napiclient: 31 | image: registry.gitlab.com/hesperos/napi/napiclient:latest 32 | build: 33 | context: . 34 | dockerfile: Dockerfile-napiclient 35 | 36 | container_name: tests_napiclient 37 | 38 | volumes: 39 | - opt-napi:/opt/napi 40 | - ..:/mnt 41 | 42 | working_dir: /mnt/tests 43 | 44 | links: 45 | - napiserver:napiserver 46 | 47 | depends_on: 48 | - napiserver 49 | 50 | volumes: 51 | opt-napi: 52 | -------------------------------------------------------------------------------- /CMakeLists.txt: -------------------------------------------------------------------------------- 1 | cmake_minimum_required(VERSION 3.0) 2 | project (napi NONE) 3 | 4 | set(NAPI_VERSION_MAJOR 2) 5 | set(NAPI_VERSION_MINOR 0) 6 | set(NAPI_VERSION_SUB 0) 7 | 8 | if (NOT NAPI_INSTALL_PREFIX) 9 | set(NAPI_INSTALL_PREFIX ${CMAKE_INSTALL_PREFIX}) 10 | endif() 11 | 12 | configure_file( 13 | napi.sh.in 14 | napi.sh 15 | @ONLY 16 | ) 17 | 18 | configure_file( 19 | subotage.sh.in 20 | subotage.sh 21 | @ONLY 22 | ) 23 | 24 | configure_file( 25 | libs/libnapi_version.sh.in 26 | libnapi_version.sh 27 | @ONLY 28 | ) 29 | 30 | # installation of the main entry point 31 | install(FILES 32 | "${PROJECT_BINARY_DIR}/napi.sh" 33 | "${PROJECT_BINARY_DIR}/subotage.sh" 34 | DESTINATION bin 35 | PERMISSIONS 36 | OWNER_READ OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE 37 | ) 38 | 39 | # install libraries 40 | install(DIRECTORY libs/ 41 | DESTINATION lib/napi 42 | PATTERN "libs/*sh" 43 | PERMISSIONS 44 | OWNER_READ OWNER_WRITE GROUP_READ WORLD_READ 45 | ) 46 | 47 | # rule for installing the generated version library 48 | install(FILES "${PROJECT_BINARY_DIR}/libnapi_version.sh" 49 | DESTINATION lib/napi 50 | PERMISSIONS 51 | OWNER_READ OWNER_WRITE GROUP_READ WORLD_READ 52 | ) 53 | 54 | # install actions 55 | install(DIRECTORY actions/ 56 | DESTINATION lib/napi/actions 57 | PATTERN "libs/*sh" 58 | PERMISSIONS 59 | OWNER_READ OWNER_WRITE GROUP_READ WORLD_READ 60 | ) 61 | 62 | -------------------------------------------------------------------------------- /tests/run_integration_tests.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | updateInstallation() { 4 | local containerName="tests_napiclient" 5 | 6 | echo "Updating napi installation..." 7 | docker-compose run \ 8 | -u 0 \ 9 | --name napiclient_update \ 10 | napiclient \ 11 | napiclient/bin/update_installation.sh 12 | 13 | echo "Commiting changes..." 14 | containerId=$(docker ps -a -q -f "name=napiclient_update") 15 | docker commit "$containerId" "$containerName" 16 | 17 | # remove the container 18 | echo "Cleanup..." 19 | docker rm -f "$containerId" 20 | } 21 | 22 | usage() { 23 | echo "run_integration_tests.sh [-u]" 24 | echo "Options:" 25 | echo 26 | echo " -u - update napi installation in container and run tests" 27 | echo " -U - update napi installation in container and exit (doesn't run tests)" 28 | echo 29 | } 30 | 31 | while getopts "uUh" option; do 32 | case "$option" in 33 | u) 34 | updateInstallation 35 | ;; 36 | 37 | U) 38 | # only update, don't run tests 39 | updateInstallation 40 | exit 0 41 | ;; 42 | 43 | h) 44 | usage 45 | exit 0 46 | ;; 47 | 48 | *) 49 | echo "Unexpected argument" >/dev/stderr 50 | usage 51 | exit 1 52 | ;; 53 | esac 54 | done 55 | 56 | # run the tests 57 | docker-compose run \ 58 | --rm \ 59 | napiclient \ 60 | python -m unittest discover -vfs integration_tests 61 | -------------------------------------------------------------------------------- /tests/napitester/bin/prepare_shells.pl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | use strict; 4 | use warnings; 5 | $|++; 6 | 7 | use lib qw(./common/lib/perl5); 8 | use NetInstall; 9 | 10 | my $prefix = shift // "/opt/napi/bash"; 11 | my $baseUrl = "http://ftp.gnu.org/gnu/bash"; 12 | my @versions = ( 13 | { package => 'bash-2.04.tar.gz', args => undef }, 14 | { package => 'bash-2.05.tar.gz', args => undef }, 15 | { package => 'bash-3.0.tar.gz', args => undef }, 16 | { package => 'bash-3.1.tar.gz', args => undef }, 17 | { package => 'bash-3.2.tar.gz', args => [ '--disable-nls' ] }, 18 | { package => 'bash-4.0.tar.gz', args => undef }, 19 | { package => 'bash-4.1.tar.gz', args => undef }, 20 | { package => 'bash-4.2.tar.gz', args => undef }, 21 | { package => 'bash-4.3.tar.gz', args => undef }, 22 | ); 23 | 24 | die "Shell Source directory already exists - assuming that all sources already have been downloaded and compiled\n" 25 | if ( -e $prefix && -d $prefix ); 26 | 27 | STDOUT->autoflush(1); 28 | foreach (@versions) { 29 | my $workDir = File::Temp::tempdir( CLEANUP => 1 ); 30 | 31 | my ($version) = $_->{'package'} =~ m/^bash-(.*)?\.tar\.gz$/; 32 | my $shell = 'bash-' . $version; 33 | my $tgzPath = $workDir . '/' . $_->{'package'}; 34 | my $srcPath = $workDir . '/' . $shell; 35 | my $dstPath = $prefix . '/' . $shell; 36 | my $url = $baseUrl . '/' . $_->{'package'}; 37 | 38 | NetInstall::prepareTgz($url, $workDir, 39 | $tgzPath, $srcPath, 40 | $dstPath, \&NetInstall::automakeInstall, $_->{'args'}); 41 | } 42 | -------------------------------------------------------------------------------- /tests/unit_tests/fake/libnapi_version_fake.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.ul 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # 35 | # version for the whole bundle (napi.sh & subotage.sh) 36 | # 37 | declare -r g_revision="v0.0.0" 38 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/output.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import re 4 | 5 | class Parser(object): 6 | 7 | DELIMITER_LEN = 20 8 | 9 | def __init__(self, scriptStdout, 10 | scriptStderr, 11 | returnCode): 12 | self.scriptStdout = scriptStdout 13 | self.scriptStderr = scriptStderr 14 | self.returnCode = returnCode 15 | 16 | def parseNapiStats(self): 17 | """ 18 | Extracts napi stats from the output stream 19 | """ 20 | tokens = [ 'OK', 'UNAV', 'SKIP', 'CONV', 21 | 'COVER_OK', 'COVER_UNAV', 'COVER_SKIP', 22 | 'NFO_OK', 'NFO_UNAV', 'NFO_SKIP', 'TOTAL', 'CONV_CHARSET' ] 23 | results = {} 24 | for token in tokens: 25 | m = re.search(r'{} -> (\d+)'.format(token), 26 | self.scriptStdout) 27 | results[token.lower()] = int(m.group(1) if m else 0) 28 | return results 29 | 30 | def stdoutContains(self, regex): 31 | return re.search(regex, self.scriptStdout) 32 | 33 | def stderrContains(self, regex): 34 | return re.search(regex, self.scriptStderr) 35 | 36 | def isSuccess(self): 37 | return self.returnCode == 0 38 | 39 | def hasErrors(self): 40 | return len(self.scriptStderr) or not self.isSuccess() 41 | 42 | def printStdout(self): 43 | print "STDOUT" 44 | print self.scriptStdout 45 | print "=" * self.DELIMITER_LEN 46 | 47 | def printStderr(self): 48 | print "STDERR" 49 | print self.scriptStderr 50 | print "=" * self.DELIMITER_LEN 51 | 52 | -------------------------------------------------------------------------------- /libs/libnapi_version.sh.in: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.ul 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # 35 | # version for the whole bundle (napi.sh & subotage.sh) 36 | # 37 | declare -r g_revision="v@NAPI_VERSION_MAJOR@.@NAPI_VERSION_MINOR@.@NAPI_VERSION_SUB@" 38 | -------------------------------------------------------------------------------- /tests/unit_tests/fake/libnapi_fs_fake.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | fs_md5_SO() { 35 | md5sum 36 | } 37 | 38 | fs_stat_SO() { 39 | stat -c%s "$@" 40 | } 41 | 42 | fs_base64Decode_SO() { 43 | base64 -d 44 | } 45 | 46 | fs_7z_SO() { 47 | 7z "$@" 48 | } 49 | -------------------------------------------------------------------------------- /tests/run_unit_tests.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | RESULT="failed" 4 | EXIT_STATUS=1 5 | AWK="awk" 6 | COVERAGE=0 7 | 8 | declare -r COVERAGE_JSON_PATH="coverage/kcov-merged/coverage.json" 9 | 10 | usage() { 11 | "./run_unit_tests.sh [ OPTIONS ] " 12 | } 13 | 14 | notify() { 15 | local results="$1" 16 | local msg="Tests ${results}" 17 | 18 | [ -n "$(which notify-send)" ] && 19 | notify-send "$msg" 20 | 21 | echo "$msg" 22 | } 23 | 24 | while getopts "a:h" option; do 25 | case "$option" in 26 | "a") 27 | AWK="$OPTARG" 28 | ;; 29 | "h") 30 | usage 31 | exit 1 32 | ;; 33 | esac 34 | done 35 | shift $((OPTIND -1)) 36 | 37 | # 38 | # execute the unit tests 39 | # 40 | docker-compose run \ 41 | --rm \ 42 | napitester bash -s < 1 ); 18 | 19 | sub print_status { 20 | my ($msg, $coderef, $expected) = @_; 21 | print $msg . " ... "; 22 | my $retval = &{$coderef}(); 23 | print $retval == $expected ? "OK" : "FAIL"; 24 | print "\n"; 25 | return $retval; 26 | } 27 | 28 | sub on_success { 29 | my ($retval, $coderef, $expected) = @_; 30 | $coderef->() if $retval == $expected; 31 | } 32 | 33 | on_success( 34 | print_status( 35 | "Downloading $assets_tgz", 36 | sub { 37 | getstore( $url, $wdir . '/' . $assets_tgz ); 38 | }, 39 | 200), 40 | sub { 41 | die "Unable to create the architecture independent data directory\n" 42 | unless ( -e $assets_path || mkdir ($assets_path) ); 43 | 44 | print_status( 45 | "Unpacking assets", 46 | sub { 47 | my $archive = $wdir . '/' . $assets_tgz; 48 | my $ae = Archive::Extract->new( 49 | archive => $archive ); 50 | 51 | if ($ae->extract( to => $assets_path )) { 52 | File::Find::find(sub { chmod(0755, $_); }, ($assets_path)); 53 | return 1; 54 | } 55 | 56 | return 0; 57 | }, 58 | 1 59 | ); 60 | }, 61 | 200 62 | ); 63 | -------------------------------------------------------------------------------- /tests/integration_tests/test_help.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import napi.testcase 4 | import unittest 5 | 6 | class HelpTest(napi.testcase.NapiTestCase): 7 | 8 | def test_ifMainHelpDoesNotProduceErrors(self): 9 | """ 10 | Brief: 11 | Test if the main action doesn't produce any output on stderr 12 | 13 | Procedure: 14 | 1. Call napi.sh --help 15 | 16 | Expected Results: 17 | No output on stderr. 18 | """ 19 | self.napiExecute('--help') 20 | self.assertFalse(self.output.hasErrors()) 21 | 22 | def test_ifActionsHelpDoesNotProduceErrors(self): 23 | """ 24 | Brief: 25 | Test if none of the actions produce any output on stderr 26 | 27 | Procedure: 28 | 1. Call napi --help for all actions 29 | 30 | Expected Results: 31 | No output on stderr. 32 | """ 33 | actions = { 34 | 'search': self.napiSearch, 35 | 'subtitles': self.napiSubtitles, 36 | 'download': self.napiDownload, 37 | 'scan': self.napiScan 38 | } 39 | 40 | for action, func in actions.items(): 41 | func('--help') 42 | self.assertFalse(self.output.hasErrors()) 43 | 44 | def test_ifSubotageHelpDoesNotProduceErrors(self): 45 | """ 46 | Brief: 47 | Test if call to subotage.sh's help doesn't generate any output on stderr 48 | 49 | Procedure: 50 | 1. Call subotage.sh --help 51 | 52 | Expected Results: 53 | No output on stderr. 54 | """ 55 | self.subotageExecute('--help') 56 | self.assertFalse(self.output.hasErrors()) 57 | 58 | if __name__ == '__main__': 59 | napi.testcase.runTests() 60 | -------------------------------------------------------------------------------- /libs/libnapi_retvals.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.ul 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # success 35 | export G_RETOK=0 36 | 37 | # function failed 38 | export G_RETFAIL=255 39 | 40 | # parameter error 41 | export G_RETPARAM=254 42 | 43 | # parameter/result will cause the script to break 44 | export G_RETBREAK=253 45 | 46 | # resource unavailable 47 | export G_RETUNAV=252 48 | 49 | # no action taken 50 | export G_RETNOACT=251 51 | 52 | -------------------------------------------------------------------------------- /tests/unit_tests/fake/libnapi_logging_fake.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | logging_debug() { 35 | echo " -- FAKE ${FUNCNAME[0]} $*" 36 | } 37 | 38 | logging_info() { 39 | echo " -- FAKE ${FUNCNAME[0]} $*" 40 | } 41 | 42 | logging_msg() { 43 | echo " -- FAKE ${FUNCNAME[0]} $*" 44 | } 45 | 46 | logging_warning() { 47 | echo " -- FAKE ${FUNCNAME[0]} $*" 48 | } 49 | 50 | logging_error() { 51 | echo " -- FAKE ${FUNCNAME[0]} $*" 52 | } 53 | 54 | -------------------------------------------------------------------------------- /tests/common/lib/perl5/NetInstall.pm: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | use strict; 4 | use warnings; 5 | $|++; 6 | 7 | use Exporter(); 8 | use Carp; 9 | use LWP::Simple; 10 | use Archive::Extract; 11 | use File::Temp; 12 | 13 | package NetInstall; 14 | 15 | sub getArchive { 16 | my ($url, $dstPath) = @_; 17 | print "Downloading...[$url]\n"; 18 | my $code = LWP::Simple::getstore($url, $dstPath); 19 | print "HTTP reponse: [$code]\n"; 20 | } 21 | 22 | sub extractArchive { 23 | my ($tgzPath, $dstPath) = @_; 24 | print "Extracting...\n"; 25 | my $ae = Archive::Extract->new(archive => $tgzPath); 26 | $ae->extract(to => $dstPath) 27 | and print "Unpacked\n"; 28 | } 29 | 30 | sub install { 31 | my ($srcPath, $dstPath, $installCmd, $extraArgs) = @_; 32 | print "Building & installing...\n"; 33 | if (chdir($srcPath)) { 34 | $installCmd->($dstPath, $extraArgs); 35 | chdir; 36 | } 37 | } 38 | 39 | sub pythonInstall { 40 | system("ls -l && python ./setup.py install"); 41 | } 42 | 43 | sub cmakeInstall { 44 | system("mkdir build && cd build && cmake .. && make && make install"); 45 | } 46 | 47 | sub automakeInstall { 48 | my $dstPath = shift // ""; 49 | my $extraArgs = shift // []; 50 | my $cmd = "./configure " . 51 | (length($dstPath) ? "--prefix $dstPath ": "") . 52 | "@{ $extraArgs } " . 53 | "&& make " . 54 | "&& make install"; 55 | system($cmd); 56 | } 57 | 58 | sub prepareTgz { 59 | my ($url, $workDir, 60 | $tgzPath, $srcPath, 61 | $dstPath, $installCmd, $extraArgs) = @_; 62 | 63 | getArchive($url, $tgzPath) || 64 | die "Unable to download archive\n"; 65 | 66 | extractArchive($tgzPath, $workDir) || 67 | die "Unable to extract archive\n"; 68 | 69 | install($srcPath, $dstPath, $installCmd, $extraArgs); 70 | } 71 | 72 | 1; 73 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/mock.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python2 2 | 3 | from pretenders.client.http import HTTPMock 4 | 5 | from . import xml_result 6 | 7 | class NapiprojektMock(object): 8 | 9 | DEFAULT_ADDRESS = 'napiserver' 10 | DEFAULT_PORT = 8000 11 | 12 | def __init__(self, 13 | address = DEFAULT_ADDRESS, 14 | port = DEFAULT_PORT): 15 | self.address = address 16 | self.port = port 17 | self.http = HTTPMock(self.address, self.port) 18 | self.defaultHeaders = { 19 | 'Content-Type': 'text/xml; charset=UTF-8', 20 | } 21 | 22 | def getUrl(self): 23 | return self.http.pretend_url 24 | 25 | def getRequest(self, n = 0): 26 | return self.http.get_request(n) 27 | 28 | def programPlainRequest(self, 29 | blob = None, 30 | status = None, 31 | times = 1): 32 | 33 | body = None 34 | data = blob.getData() if blob else '' 35 | if not status: 36 | status = 200 if blob else 404 37 | 38 | self.http.when( 39 | 'GET /unit_napisy/dl.php', 40 | body = body).reply( 41 | data, 42 | status = status, 43 | headers = self.defaultHeaders, 44 | times = times) 45 | 46 | def programXmlRequest(self, 47 | media, 48 | subtitles = None, 49 | cover = None, 50 | movieDetails = None, 51 | times = 1): 52 | status = 200 53 | # pretenders matches body from the beginning of the string - due to 54 | # that body filtering at the moment is out of question 55 | body = None 56 | self.http.when( 57 | 'POST /api/api-napiprojekt3.php', 58 | body = body).reply( 59 | xml_result.XmlResult(subtitles, cover, movieDetails).toString(), 60 | status = status, 61 | headers = self.defaultHeaders, 62 | times = times) 63 | 64 | 65 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/movie_details.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python2 2 | 3 | class MovieDetails(object): 4 | """ 5 | Encapsulates most of the details that napiprojekt XML provides 6 | """ 7 | 8 | def __init__(self, 9 | title, 10 | otherTitle, 11 | year, 12 | countryPl, 13 | countryEn, 14 | genrePl, 15 | genreEn, 16 | direction, 17 | screenplay, 18 | cinematography, 19 | imdb, 20 | filmweb, 21 | fdb, 22 | stopklatka, 23 | onet, 24 | wp, 25 | rating = 0, 26 | votes = 0): 27 | self.title = title 28 | self.otherTitle = otherTitle 29 | self.year = int(year) 30 | self.countryPl = countryPl 31 | self.countryEn = countryEn 32 | self.genrePl = genrePl 33 | self.genreEn = genreEn 34 | self.direction = direction 35 | self.screenplay = screenplay 36 | self.cinematography = cinematography 37 | self.imdb = imdb 38 | self.filmweb = filmweb 39 | self.fdb = fdb 40 | self.stopklatka = stopklatka 41 | self.onet = onet 42 | self.wp = wp 43 | self.rating = rating 44 | self.votes = votes 45 | 46 | @staticmethod 47 | def makeSimple(title, 48 | year, 49 | countryPl, 50 | genrePl, 51 | direction, 52 | imdb, 53 | filmweb, 54 | fdb, 55 | stopklatka, 56 | onet, 57 | wp): 58 | return MovieDetails(title, 59 | title, 60 | year, 61 | countryPl, 62 | countryPl, 63 | genrePl, 64 | genrePl, 65 | direction, 66 | direction, 67 | direction, 68 | imdb, 69 | filmweb, 70 | fdb, 71 | stopklatka, 72 | onet, 73 | wp) 74 | -------------------------------------------------------------------------------- /libs/libnapi_hooks.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # globals 35 | 36 | ######################################################################## 37 | 38 | hooks_callHook_GV() { 39 | local subtitlesFile="${1:-}" 40 | local hook="$(sysconf_getKey_SO system.hook.executable)" 41 | 42 | [ "${hook:-none}" = "none" ] && 43 | logging_debug $LINENO $"brak skonfigurowanego hooka, ignoruje" && 44 | return $G_RETNOACT 45 | 46 | [ ! -x "${hook}" ] && { 47 | logging_error $"podany skrypt jest niedostepny (lub nie ma uprawnien do wykonywania)" "[$hook]" 48 | return $G_RETPARAM 49 | } 50 | 51 | logging_msg $"wywoluje zewnetrzny skrypt: " "[$hook]" 52 | $hook "$subtitlesFile" 53 | } 54 | 55 | # EOF 56 | -------------------------------------------------------------------------------- /tests/Dockerfile-napitester: -------------------------------------------------------------------------------- 1 | FROM ubuntu:16.04 2 | 3 | RUN apt-get update -y 4 | RUN apt-get install -y --fix-missing \ 5 | pkg-config 6 | RUN apt-get install -y \ 7 | autoconf \ 8 | automake \ 9 | binutils-dev \ 10 | bison \ 11 | busybox \ 12 | cmake \ 13 | cmake-data \ 14 | flex \ 15 | g++-multilib \ 16 | gawk \ 17 | gcc-multilib \ 18 | gettext \ 19 | install-info \ 20 | libarchive-extract-perl \ 21 | libav-tools \ 22 | libcurl4-openssl-dev \ 23 | libdw-dev \ 24 | libelf-dev \ 25 | libssl-dev \ 26 | libwww-perl \ 27 | make \ 28 | mawk \ 29 | mediainfo \ 30 | mplayer2 \ 31 | ncurses-dev \ 32 | original-awk \ 33 | p7zip-full \ 34 | patch \ 35 | python-minimal \ 36 | python-pip \ 37 | python-setuptools \ 38 | shunit2 \ 39 | sudo \ 40 | texinfo \ 41 | wget \ 42 | zlib1g \ 43 | zlib1g-dev 44 | 45 | RUN apt-get install -y \ 46 | jq 47 | 48 | # set-up environment 49 | ENV NAPITESTER_HOME /home/napitester 50 | ENV NAPITESTER_BIN $NAPITESTER_HOME/bin 51 | ENV NAPITESTER_OPT /opt/napi 52 | ENV NAPITESTER_TESTDATA $NAPITESTER_OPT/testdata 53 | ENV NAPITESTER_SHELLS $NAPITESTER_OPT/bash 54 | ENV INSTALL_DIR /tmp/install 55 | 56 | RUN useradd -m -U napitester -d $NAPITESTER_HOME 57 | RUN usermod -a -G sudo napitester 58 | RUN mkdir -p $INSTALL_DIR 59 | 60 | # setup shells and test assets 61 | ADD common $INSTALL_DIR/common 62 | ADD napitester $INSTALL_DIR/napitester 63 | WORKDIR $INSTALL_DIR 64 | RUN ./napitester/bin/prepare_kcov.pl 65 | RUN ./napitester/bin/prepare_scpmocker.pl 66 | RUN ./napitester/bin/prepare_shells.pl $NAPITESTER_SHELLS 67 | RUN ./napitester/bin/prepare_assets.pl $NAPITESTER_TESTDATA 68 | 69 | # allow members of sudo group to execute sudo without password 70 | RUN echo "%sudo ALL=(ALL:ALL) NOPASSWD:ALL" > /etc/sudoers.d/nopasswd 71 | RUN chmod 0440 /etc/sudoers.d/nopasswd 72 | 73 | # switch to test user 74 | WORKDIR $NAPITESTER_HOME 75 | USER napitester 76 | RUN mkdir -p $NAPITESTER_BIN 77 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/scpmocker.py: -------------------------------------------------------------------------------- 1 | import os 2 | import subprocess 3 | 4 | 5 | class ScpMocker(object): 6 | """ 7 | This class interfaces to scpmocker - a programmable command mock. 8 | """ 9 | def __init__(self, scpMockerPath, sandboxPath): 10 | self.scpMockerPath = scpMockerPath 11 | self.sandboxPath = sandboxPath 12 | self.binPath = os.path.join(self.sandboxPath, 'bin') 13 | self.dbPath = os.path.join(self.sandboxPath, 'db') 14 | 15 | 16 | def __enter__(self): 17 | os.mkdir(self.binPath) 18 | os.mkdir(self.dbPath) 19 | self.envOrig = os.environ.copy() 20 | 21 | os.environ["PATH"] = ':'.join((self.binPath, os.environ["PATH"])) 22 | os.environ["SCPMOCKER_BIN_PATH"] = self.binPath 23 | os.environ["SCPMOCKER_DB_PATH"] = self.dbPath 24 | 25 | return self 26 | 27 | def __exit__(self, *args): 28 | os.environ.clear() 29 | os.environ.update(self.envOrig) 30 | 31 | def getPath(self, cmd): 32 | return os.path.join(self.binPath, cmd) 33 | 34 | def patchCmd(self, cmd): 35 | cmdPath = self.getPath(cmd) 36 | os.symlink(self.scpMockerPath, cmdPath) 37 | 38 | def getCallCount(self, cmd): 39 | inv = [ self.scpMockerPath, '-c', cmd, 'status', '-C' ] 40 | output = subprocess.check_output(inv).strip() 41 | return int(output.strip()) 42 | 43 | def getCallArgs(self, cmd, n): 44 | inv = [ self.scpMockerPath, '-c', cmd, 'status', '-A', str(n) ] 45 | output = subprocess.check_output(inv).strip() 46 | return output 47 | 48 | def program(self, cmd, stdoutStr = "", exitStatus = 0, n = 0): 49 | inv = [ self.scpMockerPath, '-c', cmd, 'program', 50 | '-e', str(exitStatus), 51 | '-s', stdoutStr, 52 | ] 53 | 54 | if n == 0: 55 | inv.append('-a') 56 | subprocess.call(inv) 57 | else: 58 | for _ in xrange(n): 59 | subprocess.call(inv) 60 | 61 | def unPatchCmd(self, cmd): 62 | cmdPath = self.getPath(cmd) 63 | try: 64 | os.unlink(cmdPath) 65 | except OSError as e: 66 | # TODO add logging? 67 | pass 68 | 69 | 70 | -------------------------------------------------------------------------------- /libs/libnapi_constants.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.ul 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # 35 | # client version which will be presented to remote services 36 | # 37 | declare -r g_napiprojektClientVersion="2.2.0.2399" 38 | 39 | # 40 | # base address of the napiprojekt service 41 | # 42 | declare -r g_napiprojektBaseUrl="${NAPIPROJEKT_BASEURL:-http://napiprojekt.pl}" 43 | 44 | # 45 | # XML API URI 46 | # 47 | declare -r g_napiprojektApi3Uri='/api/api-napiprojekt3.php' 48 | 49 | # 50 | # Legacy API URI 51 | # 52 | declare -r g_napiprojektApiLegacyUri='/unit_napisy/dl.php' 53 | 54 | # 55 | # password to napiprojekt archives 56 | # 57 | declare -r g_napiprojektPassword='iBlm8NTigvru0Jr0' 58 | 59 | # 60 | # Cover download service URI 61 | # 62 | declare -r g_napiprojektCoverUri='/okladka_pobierz.php' 63 | 64 | # 65 | # Movie catalogue search 66 | # 67 | declare -r g_napiprojektMovieCatalogueSearchUri='/ajax/search_catalog.php' 68 | 69 | -------------------------------------------------------------------------------- /INSTALL.md: -------------------------------------------------------------------------------- 1 | # Instalation 2 | 3 | **bashnapi** provides a simple install.sh script. It should be used to install the bundle. There's a reason why it doesn't use autotools, cmake or any other build system. Most of the embedded devices (routers, NASes) etc. don't have any of these installed. 4 | 5 | 6 | ## Automatic (recommended) 7 | 8 | Use the **install.sh** script to copy napi.sh & subotage.sh to the bin directory (/usr/bin - by default) and libnapi_common.sh library to a shared directory (/usr/share/napi - by default). 9 | If you want to install into directories different than defaults specify them in install.sh invocation (USE ABSOLUTE PATHS ONLY). 10 | 11 | To install napi.sh & subotage.sh under /bin and libnapi_common.sh under /shared/napi: 12 | 13 | `$ ./install.sh --bindir /bin --shareddir /shared` 14 | 15 | To install napi.sh & subotage.sh under /usr/bin (default path) and libnapi_common.sh under /my/custom/directory/napi 16 | 17 | `$ ./install.sh --shareddir /my/custom/directory` 18 | --- 19 | 20 | ## Manual (only for experts) 21 | 22 | napi.sh & subotage.sh share some common code from libnapi_common.sh. Both of them are sourcing this file. Bellow is an example installation procedure given (executables under /usr/bin, libraries under /usr/shared/napi) 23 | 24 | 1. Edit path to libnapi_common.sh in napi.sh & subotage.sh (the line numbers may slightly differ): 25 | 26 | Search for a variable NAPI_COMMON_PATH 27 | 28 | 38 29 | 39 # verify presence of the napi_common library 30 | 40 declare -r NAPI_COMMON_PATH= 31 | 32 | and set it to /usr/shared/napi 33 | 34 | 38 35 | 39 # verify presence of the napi_common library 36 | 40 declare -r NAPI_COMMON_PATH="/usr/shared/napi" 37 | 38 | 2. Place the napi.sh & subotage.sh under /usr/bin: 39 | 40 | $ cp -v napi.sh subotage.sh /usr/bin 41 | 42 | 3. Create the /usr/shared/napi directory and place the library inside of it: 43 | 44 | $ mkdir -p /usr/shared/napi 45 | $ cp -v libnapi_common.sh /usr/shared/napi 46 | 47 | 48 | **bashnapi** bundle is now installed and ready to use. 49 | --- 50 | 51 | 52 | ## 3rd_party tools required: 53 | - bash shell 54 | - wget (can be installed via Homebrew/Macports) 55 | - find tool 56 | - dd (coreutils) 57 | - md5sum (in OS X, it's md5) 58 | - cut 59 | - mktemp 60 | - 7z (if napiprojekt3 is used) (p7zip) 61 | - awk 62 | - grep 63 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/runner.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import os 4 | import subprocess 5 | import logging 6 | 7 | from . import output as OutputParser 8 | 9 | NAPIPROJEKT_BASEURL_DEFAULT = 'http://napiprojekt.pl' 10 | 11 | class Runner(object): 12 | testWrapper = "test_wrapper.sh" 13 | testWrapperPath = os.path.join( 14 | os.path.dirname(os.path.realpath(__file__)), 15 | "..", "bin", 16 | testWrapper) 17 | 18 | def __init__(self, 19 | napiprojektUrl = NAPIPROJEKT_BASEURL_DEFAULT, 20 | bash = None): 21 | self.logger = logging.getLogger() 22 | self.bash = bash if bash else '/bin/bash' 23 | self.napiprojektUrl = napiprojektUrl 24 | 25 | self._prepareEnv() 26 | 27 | def _prepareEnv(self): 28 | if self.napiprojektUrl != NAPIPROJEKT_BASEURL_DEFAULT: 29 | os.environ['NAPIPROJEKT_BASEURL'] = self.napiprojektUrl 30 | 31 | def _execute(self, executable, testTraceFilePath, *args): 32 | 33 | cmd = [ self.testWrapperPath, testTraceFilePath, 34 | self.bash, executable, ] + map(str, args) 35 | self.logger.info(cmd) 36 | process = subprocess.Popen( 37 | cmd, 38 | shell = False, 39 | bufsize = 1024, 40 | stderr = subprocess.PIPE, 41 | stdout = subprocess.PIPE) 42 | 43 | pStdout, pStderr = process.communicate() 44 | self.logger.debug('stdout: %s' % (pStdout)) 45 | self.logger.debug('stderr: %s' % (pStderr)) 46 | self.logger.info('return code: %d' % (process.returncode)) 47 | return pStdout, pStderr, process.returncode 48 | 49 | def executeNapi(self, testTraceFilePath, *args): 50 | output = self._execute('napi.sh', testTraceFilePath, *args) 51 | return OutputParser.Parser(*output) 52 | 53 | def executeSubotage(self, testTraceFilePath, *args): 54 | output = self._execute('subotage.sh', testTraceFilePath, *args) 55 | return OutputParser.Parser(*output) 56 | 57 | def scan(self, testTraceFilePath, *args): 58 | return self.executeNapi(testTraceFilePath, 59 | 'scan', '-v', '3', *args) 60 | 61 | def download(self, testTraceFilePath, *args): 62 | return self.executeNapi(testTraceFilePath, 63 | 'download', '-v', '3', *args) 64 | 65 | def subtitles(self, testTraceFilePath, *args): 66 | return self.executeNapi(testTraceFilePath, 67 | 'subtitles', '-v', '3', *args) 68 | 69 | def search(self, testTraceFilePath, *args): 70 | return self.executeNapi(testTraceFilePath, 71 | 'search', '-v', '3', *args) 72 | -------------------------------------------------------------------------------- /libs/libnapi_xml.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # 35 | # @brief extracts xml tag contents (embraced with the tag itself) 36 | # @param tag name 37 | # @param file name (optional) 38 | # 39 | xml_extractXmlTag() { 40 | local tag="$1" 41 | local filePath="${2:-/dev/stdin}" 42 | local awkScript= 43 | 44 | # embed small awk program to extract the tag contents 45 | read -d "" awkScript << EOF 46 | BEGIN { 47 | RS=">" 48 | ORS=">" 49 | } 50 | /<$tag/,/<\\\/$tag/ { print } 51 | EOF 52 | awk "$awkScript" "$filePath" 53 | } 54 | 55 | # 56 | # @brief extracts cdata contents 57 | # @param file name or none (if used as a stream filter) 58 | # 59 | xml_extractCdataTag() { 60 | local filePath="${1:-/dev/stdin}" 61 | local awkScript= 62 | 63 | # embed small awk program to extract the tag contents 64 | read -d "" awkScript << EOF 65 | BEGIN { 66 | # this can't be a string - single character delimiter (to be portable) 67 | # RS="CDATA"; 68 | FS="[\\]\\[]"; 69 | } 70 | { 71 | print \$3; 72 | } 73 | EOF 74 | awk "$awkScript" "$filePath" | tr -d '\n' 75 | } 76 | 77 | # 78 | # @brief strip xml tag 79 | # @param file name (optional - can be used as a stream filter) 80 | # 81 | xml_stripXmlTag() { 82 | local filePath="${1:-/dev/stdin}" 83 | sed 's/<[^>]\+>//g' "$filePath" 84 | } 85 | -------------------------------------------------------------------------------- /libs/libnapi_sysconf.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # This module should be used for sharing values between modules. If a variable 35 | # has a default value which will be accessed from multiple places it should be 36 | # placed here. For module specific purposes module global variables should be 37 | # used instead. 38 | 39 | ######################################################################## 40 | 41 | # globals 42 | 43 | # 44 | # The keys should be of format: 45 | # module.category.value 46 | # 47 | 48 | declare -a ___g_sysconf_configuration=( \ 49 | "napiprojekt.subtitles.orig.prefix=ORIG_" \ 50 | "napiprojekt.subtitles.orig.delete=0" \ 51 | "napiprojekt.subtitles.extension=txt" \ 52 | "napiprojekt.subtitles.format=default" \ 53 | "napiprojekt.subtitles.encoding=default" \ 54 | "napiprojekt.cover.extension=jpg" \ 55 | "napiprojekt.cover.download=0" \ 56 | "napiprojekt.nfo.extension=nfo" \ 57 | "napiprojekt.nfo.download=0" \ 58 | "system.hook.executable=none" \ 59 | "system.forks=1" \ 60 | ) 61 | 62 | ######################################################################## 63 | 64 | sysconf_setKey_GV() { 65 | logging_debug $LINENO $"ustawiam wartosc klucza:" "[$1] -> [$2]" 66 | assoc_modifyValue_GV "$1" "$2" "___g_sysconf_configuration" 67 | # ___g_sysconf_configuration="$(assoc_modifyValue_SO \ 68 | # "$1" "$2" "${___g_sysconf_configuration[@]}")" 69 | } 70 | 71 | sysconf_getKey_SO() { 72 | assoc_lookupValue_SO "${1}" "${___g_sysconf_configuration[@]}" 73 | } 74 | 75 | ################################################################################ 76 | 77 | # EOF 78 | -------------------------------------------------------------------------------- /test_tools.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | 7 | ######################################################################## 8 | ######################################################################## 9 | ######################################################################## 10 | 11 | # Copyright (C) 2017 Tomasz Wisniewski aka 12 | # DAGON 13 | # 14 | # http://github.com/dagon666 15 | # http://pcarduino.blogspot.co.ul 16 | # 17 | # 18 | # This program is free software: you can redistribute it and/or modify 19 | # it under the terms of the GNU General Public License as published by 20 | # the Free Software Foundation, either version 3 of the License, or 21 | # (at your option) any later version. 22 | # 23 | # This program is distributed in the hope that it will be useful, 24 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 25 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 26 | # GNU General Public License for more details. 27 | # 28 | # You should have received a copy of the GNU General Public License 29 | # along with this program. If not, see . 30 | 31 | ######################################################################## 32 | ######################################################################## 33 | ######################################################################## 34 | 35 | 36 | test_tool() { 37 | eval "$2" > /tmp/tmp_result 2> /tmp/tmp_err 38 | 39 | if [[ -s /tmp/tmp_err ]]; then 40 | echo "[ERROR] $1" 41 | cat /tmp/tmp_err 42 | elif [[ -s /tmp/tmp_result ]]; then 43 | echo "[OK] $1" 44 | else 45 | echo "[ERROR] $1: Unexpected result" 46 | fi 47 | 48 | rm -rf /tmp/err 49 | rm -rf /tmp/tmp_result 50 | } 51 | 52 | test_local_array() { 53 | declare -a a=( 1 2 3 4 5 ) 54 | local i=0 55 | 56 | for i in $(seq 0 4); do 57 | echo $i >> /tmp/test.la 58 | done 59 | 60 | if [[ $(wc -l /tmp/test.la | cut -d ' ' -f 1) -eq 5 ]]; then 61 | echo "[OK] local arrays" 62 | else 63 | echo "[ERROR] Your shell has problems with local arrays" 64 | fi 65 | rm /tmp/test.la 66 | } 67 | 68 | 69 | test_tool "cut" "echo abc 123 efg | cut -d ' ' -f 1" 70 | test_tool "sed" "echo abc 123 efg | sed 's/^[a-z]*//'" 71 | test_tool "head" "echo abc\n123\nefg | head -n 1" 72 | test_tool "awk" "echo abc 123 efg | awk '{ print $1 }'" 73 | test_tool "grep" "echo abc 123 efg | grep -i 'abc'" 74 | test_tool "tr" "echo abc 123 efg | tr 'abc' 'xxx' | grep -i 'xxx'" 75 | test_tool "printf" "printf '%s' abcdef | grep -i 'abc'" 76 | test_tool "wget" "wget --help | grep -i 'wget'" 77 | test_tool "find" "mkdir -p /tmp/test/xxx && find /tmp/test -type d -name xxx | grep -i 'xxx' ; rm -rf /tmp/test" 78 | test_tool "seq" "seq 32 64 | grep -i 50" 79 | test_tool "dd" "dd if=/dev/urandom count=32 bs=1k of=/tmp/test.dd 2> /dev/null && stat /tmp/test.dd && rm /tmp/test.dd" 80 | test_tool "iconv" "echo x | iconv" 81 | test_tool "mktemp" "mktemp -t tmp.XXXXXXXX" 82 | test_tool "file" "file /bin/bash" 83 | 84 | # other 85 | test_local_array 86 | -------------------------------------------------------------------------------- /CodingRules.md: -------------------------------------------------------------------------------- 1 | # Functions 2 | 3 | Every new function must have a short description header similar to the doxygen 4 | header. 5 | 6 | Example: 7 | # 8 | # @brief short description of the function's purpose 9 | # 10 | # Detailed description (optional) 11 | # 12 | # @param (description of the parameter) 13 | # @return (description of return value) 14 | # 15 | 16 | Don't use bash's _function_ keyword in function declarations. It's completely 17 | unnecessary and may cause portability problems. 18 | 19 | ## Function names 20 | 21 | 1. Private functions (not to be used outside) should be prefixed with \_ 22 | (underscore). 23 | 2. The function name convention for libraries is following: 24 | 25 | [_]functionNameInCamelCase[_ReturnValueIndicator]() 26 | 27 | Output Indicators: 28 | - `SO` - function returns data through standard output 29 | - `SE` - function return data through standard error 30 | - `GV` - function modifies non-private global variables 31 | 32 | Examples: 33 | 34 | * A private function from system library returning data through standard output 35 | 36 | _system_PrintSomething_SO 37 | 38 | # Variables 39 | 40 | If not absolutely necessary, avoid global variables. If shared state is needed 41 | and you need more than one global variable, think about using an array - just 42 | to minimize global namespace pollution. 43 | 44 | Usage of global variables in functions is permitted if they belong to the same 45 | module/library. Otherwise - they should be passed as positional arguments. 46 | 47 | ## Variable names 48 | 49 | First of all, variable names should be in *English* only!. 50 | 51 | All variables should maintain the `camelCase` naming convention. 52 | 53 | Examples: 54 | 55 | myVar=123 56 | counter=0 57 | myString="someString" 58 | 59 | Global variables should be prefixed with a "g\_" prefix. 60 | 61 | All module private global variables should be prefixed with triple \_ 62 | (underscore) 63 | 64 | All variables that are exported to environment additionally should be ALL CAPS. 65 | 66 | [___][(g|G)_] 67 | 68 | Examples: 69 | 70 | g_myGlobalVariable 71 | g_someVar 72 | g_array=( 1 2 3 ) 73 | ___g_privateVariable 74 | G_EXPORTEDGLOBALVARIABLE 75 | 76 | Illegal (logically incorrect): 77 | 78 | ___G_EXPORTEDVARIABLE 79 | 80 | All static string values should be prefixed with a dollar sign $. That's for 81 | translation purposes. Read more about localization in 82 | [BashFAQ](http://mywiki.wooledge.org/BashFAQ/098) 83 | 84 | # Indendation 85 | 86 | All new files should include the vim formatting auto configuration: 87 | 88 | # vim: ts=4 shiftwidth=4 expandtab 89 | 90 | Quick summary: 91 | - Spaces over tabs. 92 | - An indent is 4 spaces wide. 93 | - K&R braces style is preferred. 94 | - No trailing white characteres. 95 | - *Only* UNIX line ends. 96 | 97 | # Bash 98 | 99 | ## Syntax 100 | 101 | Check the code with [shellcheck](https://www.shellcheck.net/) whenever 102 | possible, especially if the change is big. 103 | -------------------------------------------------------------------------------- /tests/integration_tests/test_charset.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import re 4 | 5 | import napi.fs 6 | import napi.sandbox 7 | import napi.subtitles 8 | import napi.testcase 9 | 10 | class CharsetConversionTest(napi.testcase.NapiTestCase): 11 | 12 | def _commonCharsetTest(self, charset): 13 | media = None 14 | with napi.sandbox.Sandbox() as sandbox: 15 | media = self.videoAssets.prepareRandomMedia(sandbox) 16 | 17 | # program http mock 18 | self.napiMock.programXmlRequest( 19 | media, 20 | napi.subtitles.CompressedSubtitles.fromString( 21 | media['asset'], "test subtitles")) 22 | 23 | self.napiScan('--stats', '-C', charset, media['path']) 24 | stats = self.output.parseNapiStats() 25 | 26 | self.assertEquals(1, stats['ok']) 27 | self.assertEquals(1, stats['total']) 28 | self.assertEquals(1, stats['conv_charset']) 29 | self.assertEquals(0, stats['unav']) 30 | self.assertTrue(napi.fs.Filesystem(media).subtitlesExists()) 31 | 32 | 33 | def test_ifIconvInvocationIsCorrectForUtf8(self): 34 | """ 35 | Brief: 36 | Test if charset conversion to utf8 succeeds for the subtitles file 37 | 38 | Procedure: 39 | 1. Prepare a media file 40 | 2. Program napiprojekt.pl mock to respond with success XmlResponse 41 | 3. Call napi -C utf8 42 | 43 | Expected Results: 44 | The charset should be successfully converted 45 | 46 | """ 47 | self._commonCharsetTest('utf8') 48 | 49 | def test_ifConvertsToIso88592(self): 50 | """ 51 | Brief: 52 | Test if charset conversion to ISO_8859-2 succeeds for the subtitles file 53 | 54 | Procedure: 55 | 1. Prepare a media file 56 | 2. Program napiprojekt.pl mock to respond with success XmlResponse 57 | 3. Call napi -C ISO_8859-2 58 | 59 | Expected Results: 60 | The charset should be successfully converted 61 | 62 | """ 63 | self._commonCharsetTest('ISO_8859-2') 64 | 65 | def test_ifHandlesConversionFailureCorrectly(self): 66 | """ 67 | Brief: 68 | Procedure: 69 | Expected Results: 70 | """ 71 | media = None 72 | charset = 'completely-unsupported-charset-from-space' 73 | self.isStderrExpected = True 74 | 75 | with napi.sandbox.Sandbox() as sandbox: 76 | media = self.videoAssets.prepareRandomMedia(sandbox) 77 | 78 | # program http mock 79 | self.napiMock.programXmlRequest( 80 | media, 81 | napi.subtitles.CompressedSubtitles.fromString( 82 | media['asset'], "test subtitles")) 83 | 84 | self.napiScan('--stats', '-C', charset, media['path']) 85 | self.assertTrue(self.output.stderrContains( 86 | re.compile(r'konwersja kodowania niepomyslna'))) 87 | 88 | self.assertTrue(napi.fs.Filesystem(media).subtitlesExists()) 89 | 90 | 91 | if __name__ == '__main__': 92 | napi.testcase.runTests() 93 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/testcase.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import os 4 | import sys 5 | import unittest 6 | import uuid 7 | import logging 8 | 9 | from . import assets 10 | from . import mock 11 | from . import runner 12 | 13 | class NapiTestCase(unittest.TestCase): 14 | SHELL = "/bin/bash" 15 | 16 | def setUp(self): 17 | self.logger = logging.getLogger() 18 | self.napiMock = mock.NapiprojektMock() 19 | self.napiprojektUrl = self.napiMock.getUrl() 20 | self.runner = runner.Runner(self.napiprojektUrl, self.SHELL) 21 | self.assetsPath = os.path.join( 22 | os.environ.get('NAPICLIENT_TESTDATA', '/opt/napi/testdata'), 23 | 'testdata') 24 | 25 | self.videoAssetsPath = os.path.join( 26 | self.assetsPath, 'media') 27 | 28 | self.subtitlesAssetsPath = os.path.join( 29 | self.assetsPath, 'subtitles') 30 | 31 | self.videoAssets = assets.VideoAssets(self.videoAssetsPath) 32 | self.subtitlesAssets = assets.SubtitlesAssets(self.subtitlesAssetsPath) 33 | 34 | # should be used to store the napi output 35 | self.output = None 36 | self.isStderrExpected = False 37 | 38 | # trace files 39 | self.testTraceFilePaths = [] 40 | 41 | def tearDown(self): 42 | if (self.output and 43 | self.output.hasErrors() and 44 | not self.isStderrExpected): 45 | self.output.printStdout() 46 | self.output.printStderr() 47 | else: 48 | self._cleanupTraceFiles() 49 | 50 | def _createTraceFilePath(self): 51 | self.testTraceFile = "testrun_{}_{}.log".format( 52 | self.id(), uuid.uuid4().hex) 53 | testTraceFilePath = self.testTraceFile 54 | self.testTraceFilePaths.append(testTraceFilePath) 55 | return testTraceFilePath 56 | 57 | def _cleanupTraceFiles(self): 58 | for traceFile in self.testTraceFilePaths: 59 | if os.path.exists(traceFile): 60 | os.remove(traceFile) 61 | 62 | def napiExecute(self, *args): 63 | self.output = self.runner.executeNapi( 64 | self._createTraceFilePath(), 65 | *args) 66 | 67 | def subotageExecute(self, *args): 68 | self.output = self.runner.executeSubotage( 69 | self._createTraceFilePath(), 70 | *args) 71 | 72 | def napiScan(self, *args): 73 | self.output = self.runner.scan( 74 | self._createTraceFilePath(), 75 | *args) 76 | 77 | def napiDownload(self, *args): 78 | self.output = self.runner.download( 79 | self._createTraceFilePath(), 80 | *args) 81 | 82 | def napiSubtitles(self, *args): 83 | self.output = self.runner.subtitles( 84 | self._createTraceFilePath(), 85 | *args) 86 | 87 | def napiSearch(self, *args): 88 | self.output = self.runner.search( 89 | self._createTraceFilePath(), 90 | *args) 91 | 92 | def runTests(): 93 | # inject shell 94 | if len(sys.argv) > 1: 95 | NapiTestCase.SHELL = sys.argv.pop() 96 | 97 | # run unit tests 98 | unittest.main() 99 | 100 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/blob.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python2 2 | 3 | import base64 4 | import hashlib 5 | import logging 6 | import os 7 | import uuid 8 | import subprocess 9 | import tempfile 10 | 11 | from . import sandbox 12 | 13 | class Blob(object): 14 | """ 15 | Abstraction around data blob obtained from napi 16 | """ 17 | def __init__(self, objectId, data): 18 | self.objectId = objectId 19 | self.data = data 20 | self.logger = logging.getLogger() 21 | 22 | def getId(self): 23 | return self.objectId 24 | 25 | def getHash(self): 26 | md5 = hashlib.md5() 27 | md5.update(self.data) 28 | return md5.hexdigest() 29 | 30 | def getSize(self): 31 | return len(self.data) 32 | 33 | def getData(self): 34 | return self.data 35 | 36 | def getBase64(self): 37 | return base64.b64encode(self.data) 38 | 39 | @classmethod 40 | def fromFile(cls, objectId, path): 41 | with open(path, 'rb') as inputFile: 42 | return cls(objectId, inputFile.read()) 43 | 44 | @classmethod 45 | def fromString(cls, objectId, data): 46 | return cls(objectId, data) 47 | 48 | 49 | class CompressedBlob(Blob): 50 | """ 51 | Abstraction around compressed data blob 52 | """ 53 | def __init__(self, objectId, data, fileName, password = None): 54 | """ 55 | Compress provided data 56 | data - data to be compressed 57 | fileName - name of the compressed file in archive 58 | """ 59 | super(CompressedBlob, self).__init__(objectId, data) 60 | self.uncompressedData = self.data 61 | 62 | with sandbox.Sandbox() as sbx: 63 | filePath = os.path.join(sbx.path, fileName) 64 | 65 | # create a file with data in the sandbox 66 | with open(filePath, 'w+') as dataFile: 67 | dataFile.write(data) 68 | 69 | # create an archive name 70 | archiveName = os.path.join(sbx.path, 71 | uuid.uuid4().hex) 72 | 73 | # capture output 74 | with tempfile.TemporaryFile() as cStdout, tempfile.TemporaryFile() as cStderr: 75 | try: 76 | cmd7z = [ '7z', 'a' ] 77 | if password: 78 | cmd7z.append("-p%s" % (password)) 79 | cmd7z.extend(['-m01=bzip2', archiveName, filePath]) 80 | 81 | subprocess.check_call(cmd7z, 82 | stdout=cStdout, stderr=cStderr) 83 | 84 | # read back the archive file 85 | with open(archiveName + '.7z', 'rb') as archiveFile: 86 | self.data = archiveFile.read() 87 | 88 | except subprocess.CalledProcessError as e: 89 | self.logger.error("Unable to prepare subtitles: " + 90 | str(e)) 91 | self.logger.info(cStdout.read()) 92 | self.logger.info(cStderr.read()) 93 | 94 | else: 95 | self.logger.info("OK") 96 | 97 | def getUncompressedData(self): 98 | return self.uncompressedData 99 | 100 | def getUncompressedSize(self): 101 | return len(self.uncompressedData) 102 | 103 | def getUncompressedBase64(self): 104 | return base64.b64encode(self.uncompressedData) 105 | 106 | -------------------------------------------------------------------------------- /libs/libnapi_argv.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # globals 35 | 36 | ___g_argvOutputHandlerType= 37 | ___g_argvOutputHandler= 38 | ___g_argvErrorMsg= 39 | 40 | ######################################################################## 41 | 42 | # 43 | # @brief option dispatcher stub 44 | # 45 | argv_nullDispatcher() { 46 | return $G_RETOK 47 | } 48 | 49 | # 50 | # @brief generic option parser 51 | # 52 | argv_argvParser_GV() { 53 | local argvDispatcher="${1:-argv_nullDispatcher}" 54 | shift 55 | 56 | # command line arguments parsing 57 | while [ $# -gt 0 ]; do 58 | ___g_argvOutputHandlerType= 59 | ___g_argvOutputHandler= 60 | ___g_argvErrorMsg= 61 | 62 | logging_debug $LINENO $"dispatcher opcji:" \ 63 | "[$argvDispatcher]" 64 | 65 | "$argvDispatcher" "$@" || { 66 | local s=$? 67 | logging_debug $LINENO $"blad dispatchera opcji" 68 | return $s 69 | } 70 | 71 | logging_debug $LINENO $"wywoluje handler" \ 72 | "[$___g_argvOutputHandler]" $"typu" \ 73 | "[$___g_argvOutputHandlerType]" 74 | 75 | shift 76 | 77 | # check if there's anything to do 78 | [ -z "$___g_argvOutputHandlerType" ] || 79 | [ -z "$___g_argvOutputHandler" ] && continue 80 | 81 | # check if there's any argument provided 82 | # shellcheck disable=SC2086 83 | [ -z "$1" ] && { 84 | logging_error "$msg" 85 | return $G_RETFAIL 86 | } 87 | 88 | case "$___g_argvOutputHandlerType" in 89 | "var"|"variable"|"v") 90 | # set the variable's value 91 | eval "${___g_argvOutputHandler}=\$1" 92 | ;; 93 | "func"|"function"|"f") 94 | logging_debug $LINENO $"wywoluje setter" "[$___g_argvOutputHandler]" 95 | # I want the splitting to occur to be able to hard-code 96 | # arguments to function 97 | $___g_argvOutputHandler "$1" 98 | ;; 99 | *) 100 | ;; 101 | esac 102 | shift 103 | done 104 | } 105 | 106 | # EOF 107 | -------------------------------------------------------------------------------- /tests/unit_tests/libnapi_argv_test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # module dependencies 35 | . ../../libs/libnapi_retvals.sh 36 | 37 | # fakes/mocks 38 | . fake/libnapi_logging_fake.sh 39 | 40 | # module under test 41 | . ../../libs/libnapi_argv.sh 42 | 43 | _test_argvOptionSetter() { 44 | globalVariableModifiedBySetter="$1" 45 | } 46 | 47 | _test_argvDispatcher() { 48 | case "$1" in 49 | "--return-value") 50 | return "$2" 51 | ;; 52 | 53 | "--set-var") 54 | ___g_argvOutputHandlerType="var" 55 | ___g_argvOutputHandler="globalVariable" 56 | ___g_argvErrorMsg="no value for variable" 57 | ;; 58 | 59 | "--call-setter") 60 | ___g_argvOutputHandlerType="func" 61 | ___g_argvOutputHandler="_test_argvOptionSetter" 62 | ___g_argvErrorMsg="no value for option setter" 63 | ;; 64 | esac 65 | } 66 | 67 | test_that_argvParser_GV_forwardsArgvReturnValue() { 68 | local rv=123 69 | argv_argvParser_GV _test_argvDispatcher "--return-value" "$rv" 70 | assertEquals "checking forwarded dispatcher's return value" "$rv" $? 71 | } 72 | 73 | test_that_argvParser_GV_setsTheGlobalVariable() { 74 | local newValue="new_value_for_global with white characters" 75 | globalVariable="empty" 76 | 77 | assertNotEquals "checking global variable initial value" \ 78 | "$globalVariable" "$newValue" 79 | 80 | argv_argvParser_GV _test_argvDispatcher "--set-var" "$newValue" 81 | 82 | assertEquals "checking globalVariable value" \ 83 | "$globalVariable" "$newValue" 84 | } 85 | 86 | test_that_argvParser_GV_callsTheSetter() { 87 | local newValue="new value for setter" 88 | argv_argvParser_GV _test_argvDispatcher "--call-setter" "$newValue" 89 | assertEquals "checking globalVariable value set by setter" \ 90 | "$newValue" "$globalVariableModifiedBySetter" 91 | } 92 | 93 | test_that_argvParser_GV_failsIfNoValueForOption() { 94 | argv_argvParser_GV _test_argvDispatcher "--call-setter" "$newValue" 95 | 96 | assertEquals "checking for return value" \ 97 | $G_RETFAIL "$?" 98 | } 99 | 100 | # shunit call 101 | . shunit2 102 | -------------------------------------------------------------------------------- /tests/README.md: -------------------------------------------------------------------------------- 1 | # Test suite 2 | 3 | bashnapi comes with a comprehensive test suite and test environment. In order 4 | to use it you'll need [Docker](https://www.docker.com). 5 | 6 | ## Containers 7 | 8 | In order to support the environment you'll need to build images for the 9 | _Docker_ containers. The following _Docker_ files have been provided: 10 | 11 | - `Dockerfile-napitester` - an image containing all the dependencies and 12 | libraries for unit testing. 13 | 14 | - `Dockerfile-napiserver` - an image running 15 | [Pretenders](https://github.com/pretenders/pretenders) used for integration 16 | testing and acting as [napiprojekt.pl](http://napiprojekt.pl) mock. 17 | 18 | - `Dockerfile-napiclient` - this image is extending a napi image from the main 19 | directory - (you'll have to build it first), and is used for integration 20 | testing. It contains an installation of napi.sh along with integration tests 21 | dependencies. 22 | 23 | ## Preparations 24 | 25 | Assuming that the current working directory is the root of the project, build 26 | the napi _Docker_ image: 27 | 28 | docker build -t napi . 29 | 30 | Once that's done, proceed to the `tests` directory to build the rest of the 31 | images: 32 | 33 | cd tests 34 | docker-compose build 35 | 36 | If the last step was successful, all the required images have been built and 37 | are ready to use. 38 | 39 | ## Unit tests 40 | 41 | To run all unit tests, simply invoke 42 | 43 | ./run_unit_tests.sh 44 | 45 | in the `tests` directory. It's possible to run a selected test only as well. 46 | Just provide the file name: 47 | 48 | ./run_unit_tests.sh libnapi_http_test.sh 49 | 50 | Each unit tests execution generates coverage report which can be found in 51 | `tests/converage` directory. Navigate your browser there to get more details. 52 | 53 | ## Integration tests 54 | 55 | Integration test suite will start a dedicated container running python 56 | pretenders to mock napiprojekt.pl. The test will run in a separate container. To run the tests just invoke: 57 | 58 | ./run_integration_tests.sh 59 | 60 | If you change any of napi code, these changes will have to be incorporated into 61 | Docker container as well, for the test suite to pick it up. In order to quickly 62 | do that without rebuilding the images, just invoke: 63 | 64 | ./run_integration_tests.sh -u 65 | 66 | ### Running integration tests manually 67 | 68 | It's possible to execute only selected test fixture: 69 | 70 | docker-compose run --rm napiclient python -m unittest integration_tests.test_formats.FormatsConversionTest 71 | 72 | ... or a test case: 73 | 74 | docker-compose run --rm napiclient python -m unittest integration_tests.test_formats.FormatsConversionTest.test_ifSubotageDetectsFormatsCorrectly 75 | 76 | 77 | In order to increase verbosity of the test suite, one can define an 78 | environmental variable: `NAPI_INTEGRATION_TESTS_LOGLEVEL=1`. When run manually 79 | this can be done with docker like so: 80 | 81 | docker-compose run --rm -e NAPI_INTEGRATION_TESTS_LOGLEVEL=1 napiclient python -m unittest integration_tests.test_formats.FormatsConversionTest.test_ifSubotageDetectsFormatsCorrectly 82 | 83 | The integration tests suite contains some long running tests which are skipped 84 | by default, as the total execution time may be longer than an hour. In order to 85 | enable them, an environment variable `NAPI_INTEGRATION_TESTS_LONG_ENABLED=1` 86 | should be defined. When ran manually this can be done exactly the same way as 87 | in the previous example: 88 | 89 | docker-compose run --rm -e NAPI_INTEGRATION_TESTS_LOGLEVEL=1 -e NAPI_INTEGRATION_TESTS_LONG_ENABLED=1 napiclient python -m unittest integration_tests.test_formats.FormatsConversionTest.test_ifSubotageDetectsFormatsCorrectly 90 | -------------------------------------------------------------------------------- /tests/unit_tests/mock/scpmocker.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # A set of helper functions and wrappers for scpmocker 35 | 36 | ######################################################################## 37 | 38 | declare -r \ 39 | SCPMOCKER_FUNCTION_CMD_PREFIX="func_" 40 | declare -r \ 41 | SCPMOCKER_FUNCTION_ORIG_FUNC_PREFIX="___scpmocker_original_implementation." 42 | 43 | ######################################################################## 44 | 45 | scpmocker_setUp() { 46 | export SCPMOCKER_DB_PATH="$(mktemp -d -p "${SHUNIT_TMPDIR:-}")" 47 | export SCPMOCKER_BIN_PATH="$(mktemp -d -p "${SHUNIT_TMPDIR:-}")" 48 | export SCPMOCKER_PATH_ORIG="$PATH" 49 | export PATH="${SCPMOCKER_BIN_PATH}:${PATH}" 50 | } 51 | 52 | scpmocker_tearDown() { 53 | export PATH="${SCPMOCKER_PATH_ORIG}" 54 | } 55 | 56 | scpmocker_patchCommand() { 57 | ln -sf "$(which scpmocker)" "${SCPMOCKER_BIN_PATH}/${1}" 58 | } 59 | 60 | scpmocker_resetCommand() { 61 | rm -rf "${SCPMOCKER_BIN_PATH}/${1}" 62 | } 63 | 64 | _scpmocker_functionMock() { 65 | local cmd="$1" 66 | shift 67 | "${SCPMOCKER_FUNCTION_CMD_PREFIX}${cmd}" "$@" 68 | } 69 | 70 | scpmocker_patchFunction() { 71 | local funcName="$1" 72 | local cmd="${SCPMOCKER_FUNCTION_CMD_PREFIX}${funcName}" 73 | scpmocker_patchCommand "$cmd" 74 | 75 | # save original function's code under different name (only if it exists) 76 | local symbolType="$(builtin type -t "$funcName" 2>/dev/null)" 77 | [ -n "$symbolType" ] && [ "function" = "$symbolType" ] && 78 | eval "$(echo "${SCPMOCKER_FUNCTION_ORIG_FUNC_PREFIX}${funcName}()"; \ 79 | declare -f "$funcName" | tail -n +2)" 80 | 81 | # replace with a mock 82 | eval "${funcName}() { _scpmocker_functionMock \"${funcName}\" \"\$@\"; }" 83 | } 84 | 85 | scpmocker_resetFunction() { 86 | local funcName="$1" 87 | local cmd="${SCPMOCKER_FUNCTION_CMD_PREFIX}${funcName}" 88 | local origFuncName="${SCPMOCKER_FUNCTION_ORIG_FUNC_PREFIX}${funcName}" 89 | 90 | # reset the symbol 91 | unset -f "$funcName" 92 | 93 | # restore the original code 94 | local symbolType="$(builtin type -t "$origFuncName" 2>/dev/null)" 95 | 96 | [ -n "$funcName" ] && [ -n "$symbolType" ] && { 97 | # restore original function's code 98 | eval "$(echo "${funcName}()"; \ 99 | declare -f "$origFuncName" | tail -n +2)" 100 | } 101 | 102 | # reset the mock command 103 | scpmocker_resetCommand "$cmd" 104 | } 105 | -------------------------------------------------------------------------------- /libs/libnapi_http.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.ul 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # globals 35 | ___g_wget='none' 36 | 37 | ######################################################################## 38 | 39 | _http_configureWget() { 40 | local wget_cmd='wget -q -O' 41 | local wget_post=0 42 | 43 | local s_test=$(wget --help 2>&1 | grep "\-S") 44 | [ -n "$s_test" ] && wget_cmd='wget -q -S -O' 45 | 46 | local p_test=$(wget --help 2>&1 | grep "\-\-post\-") 47 | [ -n "$p_test" ] && wget_post=1 48 | 49 | # Entry format is: 50 | # | 51 | ___g_wget="$wget_post|$wget_cmd" 52 | 53 | # shellcheck disable=SC2086 54 | return $RET_OK 55 | } 56 | 57 | ######################################################################## 58 | 59 | # 60 | # @brief configure the library 61 | # 62 | http_configure_GV() { 63 | [ "$___g_wget" = 'none' ] && _http_configureWget 64 | } 65 | 66 | # 67 | # @brief execute wget 68 | # 69 | http_wget() { 70 | ${___g_wget##[0-9]*|} "$@" 71 | } 72 | 73 | # 74 | # @brief return true if wget supports POST 75 | # 76 | http_isPostRequestSupported() { 77 | [ "${___g_wget%%|*}" -eq 1 ] 78 | } 79 | 80 | # 81 | # @brief extract http status code 82 | # 83 | http_getHttpStatus() { 84 | local awkCode= 85 | read -r -d "" awkCode << 'EOF' 86 | BEGIN{ 87 | respMax = 0 88 | } 89 | 90 | /HTTP/ { 91 | m = match($0, /HTTP\/[\.0-9]* [0-9]*/); 92 | if (m) 93 | responses[respMax++] = $2 94 | } 95 | 96 | END { 97 | responseString=responses[0] 98 | for (i = 1; i >(http_getHttpStatus >&2) 138 | } 139 | 140 | ######################################################################## 141 | -------------------------------------------------------------------------------- /tests/unit_tests/libnapi_sysconf_test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # module dependencies 35 | . ../../libs/libnapi_assoc.sh 36 | . ../../libs/libnapi_retvals.sh 37 | 38 | # fakes/mocks 39 | . fake/libnapi_logging_fake.sh 40 | 41 | # module under test 42 | . ../../libs/libnapi_sysconf.sh 43 | 44 | # 45 | # tests env setup 46 | # 47 | setUp() { 48 | ___g_sysconf_configuration=() 49 | } 50 | 51 | # 52 | # tests env tear down 53 | # 54 | tearDown() { 55 | ___g_sysconf_configuration=() 56 | } 57 | 58 | test_sysconf_getKey_SO_returnsValueForExistingKey() { 59 | local key="some_key" 60 | local value="some value" 61 | ___g_sysconf_configuration=( "${key}=${value}" ) 62 | 63 | assertEquals 'check the key value' \ 64 | "$value" "$(sysconf_getKey_SO "$key")" 65 | 66 | assertEquals 'check return status on success' \ 67 | 0 $? 68 | } 69 | 70 | test_sysconf_getKey_SO_returnsValueStartingWithDelimiter() { 71 | local key="some_key" 72 | local value="===some value" 73 | ___g_sysconf_configuration=( "${key}=${value}" ) 74 | 75 | assertEquals 'check the key value' \ 76 | "$value" "$(sysconf_getKey_SO "$key")" 77 | 78 | assertEquals 'check return status on success' \ 79 | 0 $? 80 | } 81 | 82 | test_sysconf_getKey_SO_failsForNonExistingKey() { 83 | local key="some_key" 84 | local value="some value" 85 | ___g_sysconf_configuration=( "${key}=${value}" ) 86 | 87 | local returnValue= 88 | returnValue="$(sysconf_getKey_SO "non-existingKey")" 89 | 90 | assertEquals 'check return status on failure' \ 91 | "$G_RETFAIL" $? 92 | 93 | assertNotEquals 'check the key value' \ 94 | "$value" "$returnValue" 95 | } 96 | 97 | test_sysconf_setKey_GV_addsValuesWithWhiteCharacters() { 98 | local key="someKey" 99 | local value="some value with white characters" 100 | 101 | sysconf_setKey_GV "$key" "$value" 102 | 103 | assertEquals 0 $? 104 | 105 | assertEquals \ 106 | "${key}=${value}" "${___g_sysconf_configuration[*]}" 107 | 108 | assertEquals 'check the key value' \ 109 | "$value" "$(sysconf_getKey_SO "$key")" 110 | } 111 | 112 | test_sysconf_setKey_GV_modifiesAlreadyExistingKey() { 113 | local key="someKey" 114 | local originalValue="original-value" 115 | local value="some value with white characters" 116 | 117 | sysconf_setKey_GV "$key" "$originalValue" 118 | 119 | sysconf_setKey_GV "$key" "$value" 120 | 121 | assertEquals 0 $? 122 | 123 | assertNotEquals \ 124 | "$originalValue" "$(sysconf_getKey_SO "$key")" 125 | 126 | assertEquals \ 127 | "${key}=${value}" "${___g_sysconf_configuration[*]}" 128 | 129 | assertEquals 'check the key value' \ 130 | "$value" "$(sysconf_getKey_SO "$key")" 131 | } 132 | 133 | # shunit call 134 | . shunit2 135 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/fs.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python2 2 | 3 | import os 4 | import hashlib 5 | 6 | class HashedFile(object): 7 | """ 8 | Represents a file with a hash, to be able to detect if file has been 9 | modified or not. 10 | """ 11 | 12 | def __init__(self, path): 13 | self.path = path 14 | try: 15 | self.hash = self._hash() 16 | except RuntimeError as e: 17 | logging.error(str(e)) 18 | self.hash = None 19 | 20 | def _hash(self): 21 | if not os.path.exists(self.path): 22 | raise RuntimeError("Path {} doesn't exist".format(self.path)) 23 | 24 | h = hashlib.sha256() 25 | with open(self.path, "rb") as fileObj: 26 | def readChunk(): 27 | chunkSize = 2048 28 | return fileObj.read(chunkSize) 29 | 30 | for chunk in iter(readChunk, ''): 31 | h.update(chunk) 32 | 33 | return h.hexdigest() 34 | 35 | def getHash(self): 36 | return self.hash 37 | 38 | def hasChanged(self): 39 | if not self.hash: 40 | return False 41 | return self._hash() != self.hash 42 | 43 | def __eq__(self, other): 44 | return self.hash == other.hash 45 | 46 | 47 | class Filesystem(object): 48 | """ 49 | This module provides a set of utilities for napiprojekt file 50 | management 51 | """ 52 | def __init__(self, media): 53 | self.media = media 54 | self.path = self._dirname() 55 | self.noExtension = self._basename() 56 | 57 | def _dirname(self): 58 | return os.path.dirname(self.media['path']) 59 | 60 | def _basename(self): 61 | return os.path.splitext(self.media['name'])[0] 62 | 63 | def _fileExists(self, fileName): 64 | path = os.path.join(self.path, fileName) 65 | return os.path.exists(path) and os.path.getsize(path) > 0 66 | 67 | def createSubtitlesFileNames(self, 68 | prefix = None, 69 | extension = None, 70 | abbreviation = None, 71 | conversionAbbreviation = None): 72 | extensions = set([ extension ] if extension 73 | else [ 'srt', 'sub', 'txt' ]) 74 | 75 | abbreviations = [] 76 | suffixes = [] 77 | prefixes = [ self.noExtension ] 78 | 79 | if prefix: 80 | prefixes.append('_'.join((prefix, self.noExtension))) 81 | 82 | if abbreviation: 83 | abbreviations.append(abbreviation) 84 | 85 | if conversionAbbreviation: 86 | abbreviations.append(conversionAbbreviation) 87 | 88 | if abbreviation and conversionAbbreviation: 89 | abbreviations.append('.'.join( 90 | (abbreviation, conversionAbbreviation))) 91 | 92 | suffixes = [ '.'.join((abr, ext)) for ext in extensions 93 | for abr in abbreviations ] 94 | suffixes.extend(extensions) 95 | 96 | # generate all possible file names 97 | return [ '.'.join((p,s)) for p in prefixes for s in suffixes ] 98 | 99 | def createNfoFileName(self): 100 | return self.noExtension + '.nfo' 101 | 102 | def createCoverFileName(self): 103 | return self.noExtension + '.jpg' 104 | 105 | def createXmlFileName(self): 106 | return self.noExtension + '.xml' 107 | 108 | def subtitlesExists(self, prefix = None, extension = None, 109 | abbreviation = None, conversionAbbreviation = None): 110 | paths = self.getSubtitlesPaths(prefix, extension, 111 | abbreviation, conversionAbbreviation) 112 | return True if len(paths) > 0 else False 113 | 114 | def getSubtitlesPaths(self, prefix = None, extension = None, 115 | abbreviation = None, conversionAbbreviation = None): 116 | paths = [ p for p in map( 117 | lambda f: os.path.join(self.path, f), 118 | self.createSubtitlesFileNames(prefix, extension, 119 | abbreviation, conversionAbbreviation)) 120 | if os.path.exists(p) ] 121 | return paths 122 | 123 | def coverExists(self): 124 | return self._fileExists(self.createCoverFileName()) 125 | 126 | def nfoExists(self): 127 | return self._fileExists(self.createNfoFileName()) 128 | 129 | def xmlExists(self): 130 | return self._fileExists(self.createXmlFileName()) 131 | -------------------------------------------------------------------------------- /tests/unit_tests/libnapi_xml_test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # module dependencies 35 | 36 | # fakes/mocks 37 | . fake/libnapi_logging_fake.sh 38 | 39 | # module under test 40 | . ../../libs/libnapi_xml.sh 41 | 42 | # 43 | # tests env setup 44 | # 45 | setUp() { 46 | 47 | # restore original values 48 | ___g_subs_defaultExtension='txt' 49 | } 50 | 51 | # 52 | # tests env tear down 53 | # 54 | tearDown() { 55 | : 56 | } 57 | 58 | test_xml_extractXmlTag() { 59 | local contents=( "" \ 60 | "nested tag contents" \ 61 | 'data' ) 62 | 63 | local extracted= 64 | local tag= 65 | local targetTag= 66 | 67 | tag="text contents for the tag" 68 | targetTag="$tag" 69 | extracted=$(echo "$tag" | \ 70 | xml_extractXmlTag "tag") 71 | assertEquals "check output for outer most tag" \ 72 | "$targetTag" "$extracted" 73 | 74 | targetTag="text contents for the tag" 75 | tag="${targetTag}" 76 | extracted=$(echo "$tag" | \ 77 | xml_extractXmlTag "nested") 78 | 79 | assertEquals "check output for inner most tag" \ 80 | "$targetTag" "$extracted" 81 | 82 | targetTag="text contents for the tag" 83 | tag="${targetTag}" 84 | extracted=$(echo "$tag" | \ 85 | xml_extractXmlTag "nested") 86 | 87 | assertEquals "check output for inner most tag (outer most has attributes)" \ 88 | "$targetTag" "$extracted" 89 | 90 | targetTag="text contents for the tag" 91 | tag="${targetTag}" 92 | extracted=$(echo "$tag" | \ 93 | xml_extractXmlTag "nested") 94 | 95 | assertEquals "check output for deep hierarchy" \ 96 | "$targetTag" "$extracted" 97 | } 98 | 99 | test_xml_extractCdataTag_behavior() { 100 | local data="This is some arbitrary cdata content" 101 | local extracted=$(echo "![CDATA[$data]]" | xml_extractCdataTag) 102 | 103 | assertEquals "check the data extracted" \ 104 | "$data" "$extracted" 105 | 106 | local base64Payload=$(echo "$data" | base64) 107 | extracted=$(echo "![CDATA[$base64Payload]]" | xml_extractCdataTag) 108 | 109 | assertEquals "check the base64 data extracted" \ 110 | "$base64Payload" "$extracted" 111 | } 112 | 113 | test_xml_stripXmlTag() { 114 | local tags="" 115 | local data="some non-tag data" 116 | local cdata="![CDATA[${data}]]" 117 | 118 | local outermost="${data}" 119 | local stripped=$(echo "$outermost" | xml_stripXmlTag) 120 | 121 | assertEquals "check stripped output for raw data" \ 122 | "$data" "$stripped" 123 | 124 | outermost="${cdata}" 125 | stripped=$(echo "$outermost" | xml_stripXmlTag) 126 | assertEquals "check stripped output for cdata" \ 127 | "$cdata" "$stripped" 128 | 129 | outermost="${tags}" 130 | stripped=$(echo "$outermost" | xml_stripXmlTag) 131 | assertEquals "produces empty output for embedded tags" \ 132 | "" "$stripped" 133 | } 134 | 135 | # shunit call 136 | . shunit2 137 | -------------------------------------------------------------------------------- /actions/libnapi_subtitles.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | ######################################################################## 35 | 36 | # globals 37 | 38 | # 39 | # @brief aggregates all collected movie page urls 40 | # 41 | declare -a ___g_moviePageUrls=() 42 | 43 | ######################################################################## 44 | 45 | _subtitles_getSubtitlesForUrl() { 46 | local url="$1" 47 | local awkCode= 48 | local awkCode2= 49 | 50 | local subsPageUri= 51 | local subsPageUrl= 52 | 53 | read -r -d "" awkCode << 'SEARCHFORLINKAWKEOF' 54 | />napisy<\/a>/ { 55 | isHrefMatched = match($0, /href="(napisy[^"]*)"/, cHref) 56 | if (isHrefMatched) { 57 | print cHref[1] 58 | } 59 | } 60 | SEARCHFORLINKAWKEOF 61 | 62 | read -r -d "" awkCode2 << 'SEARCHFORIDSAWKEOF' 63 | BEGIN { 64 | fileSize = 0 65 | fps = 0 66 | } 67 | /bajt/ { 68 | isFileSizeMatched = match($0, /\(([0-9]+) bajt.?w\)/, fileSizeMatch) 69 | if (isFileSizeMatched) { 70 | fileSize = fileSizeMatch[1] 71 | } 72 | } 73 | /Video FPS/ { 74 | isFpsMatched = match($0, /Video FPS:[^0-9]*([0-9\.]+)/, fpsMatched) 75 | if (isFpsMatched) { 76 | fps = fpsMatched[1] 77 | } 78 | } 79 | /napiprojekt:/ { 80 | isHrefMatched = match($0, /href="(napiprojekt:[^"]*)"/, cHref) 81 | if (isHrefMatched) { 82 | printf("Rozmiar: %16s bajtow | fps: %6s | %s\n", fileSize, fps, cHref[1]) 83 | } 84 | } 85 | SEARCHFORIDSAWKEOF 86 | 87 | subsPageUri="$(http_downloadUrl_SOSE "$url" "" "" 2>/dev/null | awk "$awkCode")" 88 | subsPageUrl="${g_napiprojektBaseUrl}/${subsPageUri}" 89 | 90 | [ -n "$subsPageUri" ] && { 91 | logging_debug $LINENO $"Znaleziono link do strony napisow:" \ 92 | "[$subsPageUrl]" 93 | 94 | http_downloadUrl_SOSE "$subsPageUrl" "" "" 2>/dev/null | \ 95 | awk "$awkCode2" 96 | 97 | return $G_RETOK 98 | } 99 | 100 | return $G_RETFAIL 101 | } 102 | 103 | # 104 | # @brief attempts to extract the url to subtitles page of given movie 105 | # and extract subtitles ids for that movie 106 | # 107 | _subtitles_getSubtitles() { 108 | local url= 109 | for url in "${___g_moviePageUrls[@]}"; do 110 | logging_msg $"Przetwarzam: " "[$url]" 111 | _subtitles_getSubtitlesForUrl "$url" || return $G_RETFAIL 112 | done 113 | return $G_RETOK 114 | } 115 | 116 | _subtitles_argvDispatcher_GV() { 117 | case "$1" in 118 | *) 119 | ___g_moviePageUrls=( "${___g_moviePageUrls[@]}" "$1" ) 120 | ;; 121 | esac 122 | } 123 | 124 | # 125 | # @brief parse search specific options 126 | # 127 | # This function modifies ___g_movieTitles array 128 | # 129 | _subtitles_parseArgv_GV() { 130 | logging_debug $LINENO $"parsuje opcje akcji search" 131 | argv_argvParser_GV _subtitles_argvDispatcher_GV "$@" 132 | } 133 | 134 | # 135 | # @brief print usage description for subtitles action 136 | # 137 | subtitles_usage() { 138 | echo $"napi.sh subtitles [OPCJE] " 139 | echo 140 | echo $"OPCJE:" 141 | echo 142 | } 143 | 144 | # 145 | # @brief entry point for search action 146 | # 147 | subtitles_main() { 148 | # parse search specific options 149 | _subtitles_parseArgv_GV "$@" 150 | 151 | # search for subtitles ids 152 | _subtitles_getSubtitles 153 | } 154 | 155 | # EOF 156 | -------------------------------------------------------------------------------- /libs/libnapi_language.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # globals 35 | ___g_language_napiprojektLanguage='PL' 36 | 37 | # 38 | # napiprojekt supported languages 39 | # 40 | declare -ar ___g_napiprojektLanguages=( 'Albański' 'Angielski' 'Arabski' 'Bułgarski' \ 41 | 'Chiński' 'Chorwacki' 'Czeski' 'Duński' \ 42 | 'Estoński' 'Fiński' 'Francuski' 'Galicyjski' \ 43 | 'Grecki' 'Hebrajski' 'Hiszpanski' 'Holenderski' \ 44 | 'Indonezyjski' 'Japoński' 'Koreański' 'Macedoński' \ 45 | 'Niemiecki' 'Norweski' 'Oksytański' 'Perski' \ 46 | 'Polski' 'Portugalski' 'Portugalski' 'Rosyjski' \ 47 | 'Rumuński' 'Serbski' 'Słoweński' 'Szwedzki' \ 48 | 'Słowacki' 'Turecki' 'Wietnamski' 'Węgierski' 'Włoski' ) 49 | 50 | # 51 | # napiprojekt 2 Letter language codes 52 | # 53 | declare -ar ___g_napiprojektLanguageCodes2L=( 'SQ' 'EN' 'AR' 'BG' 'ZH' 'HR' \ 54 | 'CS' 'DA' 'ET' 'FI' 'FR' 'GL' 'EL' 'HE' 'ES' 'NL' 'ID' 'JA' 'KO' 'MK' \ 55 | 'DE' 'NO' 'OC' 'FA' 'PL' 'PT' 'PB' 'RU' 'RO' 'SR' 'SL' 'SV' 'SK' 'TR' \ 56 | 'VI' 'HU' 'IT' ) 57 | 58 | # 59 | # napiprojekt 3 Letter language codes 60 | # 61 | declare -ar ___g_napiprojektLanguageCodes3L=( 'ALB' 'ENG' 'ARA' 'BUL' 'CHI' \ 62 | 'HRV' 'CZE' 'DAN' 'EST' 'FIN' 'FRE' 'GLG' 'ELL' 'HEB' 'SPA' 'DUT' 'IND' \ 63 | 'JPN' 'KOR' 'MAC' 'GER' 'NOR' 'OCI' 'PER' 'POL' 'POR' 'POB' 'RUS' 'RUM' \ 64 | 'SCC' 'SLV' 'SWE' 'SLO' 'TUR' 'VIE' 'HUN' 'ITA' ) 65 | 66 | ######################################################################## 67 | 68 | # 69 | # @brief list all the supported languages and their respective 2/3 letter codes 70 | # 71 | language_listLanguages_SO() { 72 | local i=0 73 | while [ "$i" -lt "${#___g_napiprojektLanguages[@]}" ]; do 74 | echo "${___g_napiprojektLanguageCodes2L[$i]} /" \ 75 | "${___g_napiprojektLanguageCodes3L[$i]} - ${___g_napiprojektLanguages[$i]}" 76 | i=$(( i + 1 )) 77 | done 78 | } 79 | 80 | # 81 | # @brief verify that the given language code is supported 82 | # 83 | language_verifyLanguage_SO() { 84 | local i=0 85 | local lang="${1:-}" 86 | local langArray=( ) 87 | 88 | # shellcheck disable=SC2086 89 | [ "${#lang}" -ne 2 ] && [ "${#lang}" -ne 3 ] && return $G_RETPARAM 90 | 91 | local langArrayName="___g_napiprojektLanguageCodes${#lang}L" 92 | eval langArray=\( \${${langArrayName}[@]} \) 93 | 94 | i=$(assoc_lookupKey_SO "$lang" "${langArray[@]}") 95 | 96 | [ $? -eq $G_RETOK ] && { 97 | echo "$i" 98 | return $G_RETOK 99 | } 100 | 101 | # shellcheck disable=SC2086 102 | return $RET_FAIL 103 | } 104 | 105 | # 106 | # @brief set the language variable 107 | # @param: language index 108 | # 109 | language_normalizeLanguage_SO() { 110 | local i=${1:-0} 111 | i=$(( i + 0 )) 112 | local lang="${___g_napiprojektLanguageCodes2L[$i]}" 113 | # don't ask me why 114 | [ "$lang" = "EN" ] && lang="ENG" 115 | echo "$lang" 116 | } 117 | 118 | # 119 | # @brief set language 120 | # 121 | language_setLanguage_GV() { 122 | local idx=0 123 | ___g_language_napiprojektLanguage="${1}" 124 | 125 | if idx=$(language_verifyLanguage_SO "$___g_language_napiprojektLanguage") 126 | then 127 | ___g_language_napiprojektLanguage=$(language_normalizeLanguage_SO "$idx") 128 | logging_debug $LINENO $"jezyk skonfigurowany jako" \ 129 | "[$___g_language_napiprojektLanguage]" 130 | else 131 | logging_error $"niepoprawny kod jezyka" "[$1]" 132 | ___g_language_napiprojektLanguage="PL" 133 | fi 134 | } 135 | 136 | # 137 | # @brief return language configured 138 | # 139 | language_getLanguage_SO() { 140 | echo "$___g_language_napiprojektLanguage" 141 | } 142 | 143 | # EOF 144 | -------------------------------------------------------------------------------- /actions/libnapi_search.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | ######################################################################## 35 | 36 | # globals 37 | 38 | # 39 | # @brief aggregates all collected titles from the command line 40 | # 41 | declare -a ___g_movieTitles=() 42 | 43 | # 44 | # @brief title's release year 45 | # 46 | ___g_movieYear= 47 | 48 | # 49 | # @brief title's type (0 - movie or 1 - series) 50 | # 51 | ___g_titleType= 52 | 53 | ######################################################################## 54 | 55 | _search_argvDispatcher_GV() { 56 | case "$1" in 57 | "-y" | "--year") 58 | ___g_argvOutputHandlerType="var" 59 | ___g_argvOutputHandler='___g_movieYear' 60 | ___g_argvErrorMsg=$"nie okreslono roku" 61 | ;; 62 | 63 | "-k" | "--kind") 64 | ___g_argvOutputHandlerType="var" 65 | ___g_argvOutputHandler='___g_titleType' 66 | ___g_argvErrorMsg=$"nie okreslono typu /dev/null | \ 123 | awk -v napiBase="$g_napiprojektBaseUrl" "$awkCode" 124 | } 125 | 126 | # 127 | # @brief iterate through provided titles and get movie information for 128 | # every each of them 129 | # 130 | _search_searchTitles() { 131 | local title= 132 | for title in "${___g_movieTitles[@]}"; do 133 | logging_msg $"Wyszukuje tytul:" "[$title]" 134 | _search_searchTitle "$title" || return $G_RETFAIL 135 | done 136 | return $G_RETOK 137 | } 138 | 139 | # 140 | # @brief print usage description for search action 141 | # 142 | search_usage() { 143 | echo $"napi.sh search [OPCJE] " 144 | echo 145 | echo $"OPCJE:" 146 | echo $" -y | --year - szukaj tytulu z danego roku" 147 | echo $" -k | --kind - tytul hest filmem badz serialem" 148 | } 149 | 150 | # 151 | # @brief entry point for search action 152 | # 153 | search_main() { 154 | # parse search specific options 155 | _search_parseArgv_GV "$@" 156 | 157 | logging_debug $LINENO "Title type: ${___g_titleType}" 158 | logging_debug $LINENO "Movie year: ${___g_movieYear}" 159 | 160 | # search the movies 161 | _search_searchTitles 162 | } 163 | -------------------------------------------------------------------------------- /libs/libnapi_wrappers.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # 35 | # A set of independent wrappers for system commands. The rules of thumb for the 36 | # wrapper to be placed here are: 37 | # 38 | # - no dependency on other libraries 39 | # - doesn't fall back into responsibility of other library 40 | # - doesn't require configuration or global variables 41 | # 42 | 43 | ######################################################################## 44 | 45 | # 46 | # @brief returns numeric value even for non-numeric input 47 | # 48 | wrappers_ensureNumeric_SO() { 49 | echo $(( $1 + 0 )) 50 | } 51 | 52 | # 53 | # @brief count the number of lines in a file 54 | # 55 | wrappers_countLines_SO() { 56 | 57 | # it is being executed in a subshell to strip any leading white spaces 58 | # which some of the wc versions produce 59 | 60 | # shellcheck disable=SC2046 61 | # shellcheck disable=SC2005 62 | echo $(wc -l) 63 | } 64 | 65 | # 66 | # @brief lower-case the input 67 | # 68 | wrappers_lcase_SO() { 69 | # some old busybox implementations have problems with locales 70 | # which renders that syntax unusable 71 | # tr '[:upper:]' '[:lower:]' 72 | 73 | # deliberately reverted to old syntax 74 | # shellcheck disable=SC2021 75 | tr '[A-Z]' '[a-z]' 76 | } 77 | 78 | # 79 | # @brief upper-case the input 80 | # 81 | wrappers_ucase_SO() { 82 | # deliberately reverted to old syntax 83 | # shellcheck disable=SC2021 84 | tr '[a-z]' '[A-Z]' 85 | } 86 | 87 | # 88 | # @brief make the first letter capital 89 | # 90 | wrappers_ucaseFirst_SO() { 91 | local h="$(echo ${1:0:1} | wrappers_ucase_SO)" 92 | local t="${1:1}" 93 | echo "${h}${t}" 94 | } 95 | 96 | # 97 | # @brief get rid of the newline/carriage return 98 | # 99 | wrappers_stripNewLine_SO() { 100 | tr -d '\r\n' 101 | } 102 | 103 | # 104 | # @brief get the extension of the input 105 | # 106 | wrappers_getExt_SO() { 107 | echo "${1##*.}" 108 | } 109 | 110 | # 111 | # @brief strip the extension of the input 112 | # 113 | wrappers_stripExt_SO() { 114 | echo "${1%.*}" 115 | } 116 | 117 | # 118 | # @brief detects running system type 119 | # 120 | wrappers_getSystem_SO() { 121 | uname | wrappers_lcase_SO 122 | } 123 | 124 | # 125 | # @brief returns true if system is Darwin 126 | # 127 | wrappers_isSystemDarwin() { 128 | [ "$(wrappers_getSystem_SO)" = "darwin" ] 129 | } 130 | 131 | # 132 | # @brief determines number of available cpu's in the system 133 | # 134 | # @param system type (linux|darwin) 135 | # 136 | wrappers_getCores_SO() { 137 | local os="${1:-linux}" 138 | if wrappers_isSystemDarwin; then 139 | sysctl hw.ncpu | cut -d ' ' -f 2 140 | else 141 | grep -i processor /proc/cpuinfo | wc -l 142 | fi 143 | } 144 | 145 | # 146 | # @brief converts dos line endings to unix style line endings 147 | # 148 | wrappers_dos2unix_SO() { 149 | awk '{ sub("\r$",""); print }' 150 | } 151 | 152 | # 153 | # @brief filters out numeric and decimal dot characters 154 | # 155 | wrappers_filterNumeric_SO() { 156 | tr -d '[\r\n\.0-9]' 157 | } 158 | 159 | ################################## FLOAT CMP ################################### 160 | 161 | wrappers_floatLt() { 162 | awk -v n1="$1" -v n2="$2" 'BEGIN{ if (n1n2) exit 0; exit 1}' 168 | } 169 | 170 | 171 | wrappers_floatLe() { 172 | awk -v n1="$1" -v n2="$2" 'BEGIN{ if (n1<=n2) exit 0; exit 1}' 173 | } 174 | 175 | 176 | wrappers_floatGe() { 177 | awk -v n1="$1" -v n2="$2" 'BEGIN{ if (n1>=n2) exit 0; exit 1}' 178 | } 179 | 180 | 181 | wrappers_floatEq() { 182 | awk -v n1="$1" -v n2="$2" 'BEGIN{ if (n1==n2) exit 0; exit 1}' 183 | } 184 | 185 | wrappers_floatDiv() { 186 | awk -v n1="$1" -v n2="$2" 'BEGIN { print n1/n2 }' 2>/dev/null 187 | } 188 | 189 | wrappers_floatMul() { 190 | awk -v n1="$1" -v n2="$2" 'BEGIN { print n1*n2 }' 191 | } 192 | 193 | # EOF 194 | -------------------------------------------------------------------------------- /tests/unit_tests/libnapi_language_test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # module dependencies 35 | . ../../libs/libnapi_assoc.sh 36 | . ../../libs/libnapi_retvals.sh 37 | 38 | 39 | # fakes/mocks 40 | . fake/libnapi_logging_fake.sh 41 | . mock/scpmocker.sh 42 | 43 | # module under test 44 | . ../../libs/libnapi_language.sh 45 | 46 | setUp() { 47 | scpmocker_setUp 48 | 49 | # restore original values 50 | ___g_language_napiprojektLanguage='PL' 51 | } 52 | 53 | tearDown() { 54 | scpmocker_tearDown 55 | } 56 | 57 | test_language_listLanguages_producesOutput() { 58 | assertNotNull "Output is not empty" \ 59 | "$(language_listLanguages_SO)" 60 | } 61 | 62 | test_language_listLanguages_listsAllLanguages() { 63 | assertEquals "Matches 2 letter codes array size" \ 64 | "${#___g_napiprojektLanguageCodes2L[@]}" \ 65 | "$(language_listLanguages_SO | wc -l)" 66 | 67 | assertEquals "Matches 3 letter codes array size" \ 68 | "${#___g_napiprojektLanguageCodes3L[@]}" \ 69 | "$(language_listLanguages_SO | wc -l)" 70 | 71 | assertEquals "Matches language array size" \ 72 | "${#___g_napiprojektLanguages[@]}" \ 73 | "$(language_listLanguages_SO | wc -l)" 74 | } 75 | 76 | test_language_verifyLanguage_verifiesAllSupported2LCodes() { 77 | local key= 78 | local lang= 79 | 80 | for k in "${!___g_napiprojektLanguageCodes2L[@]}"; do 81 | lang="${___g_napiprojektLanguageCodes2L[$k]}" 82 | key=$(language_verifyLanguage_SO "$lang") 83 | assertTrue "verify exit status" $? 84 | 85 | assertEquals "verify language [$lang]" \ 86 | "$k" "$key" 87 | done 88 | } 89 | 90 | test_language_verifyLanguage_verifiesAllSupported3LCodes() { 91 | local key= 92 | local lang= 93 | 94 | for k in "${!___g_napiprojektLanguageCodes3L[@]}"; do 95 | lang="${___g_napiprojektLanguageCodes3L[$k]}" 96 | key=$(language_verifyLanguage_SO "$lang") 97 | assertTrue "verify exit status" $? 98 | 99 | assertEquals "verify language [$lang]" \ 100 | "$k" "$key" 101 | done 102 | } 103 | 104 | test_language_verifyLanguage_failsForUnknownLanguages() { 105 | local key= 106 | local lang= 107 | local langs=( 'FAKE' 'OT' 'AB' 'DEF' 'XX' ) 108 | 109 | for k in "${!langs[@]}"; do 110 | lang="${langs[$k]}" 111 | key=$(language_verifyLanguage_SO "$lang") 112 | assertFalse "verify exit status" $? 113 | 114 | assertNull "verify language [$lang]" \ 115 | "$key" 116 | done 117 | } 118 | 119 | test_language_normalizeLanguage_normalizes2LCodes() { 120 | local langOutput= 121 | local lang= 122 | 123 | for k in "${!___g_napiprojektLanguageCodes2L[@]}"; do 124 | lang="${___g_napiprojektLanguageCodes2L[$k]}" 125 | langOutput=$(language_normalizeLanguage_SO "$k") 126 | assertTrue "verify exit status" $? 127 | 128 | [ "$lang" = "EN" ] && lang="ENG" 129 | 130 | assertEquals "normalized code [$lang]" \ 131 | "$lang" "$langOutput" 132 | done 133 | } 134 | 135 | test_language_getLanguage_echoesGlobalValue() { 136 | for l in {A..Z}{A..Z}; do 137 | ___g_language_napiprojektLanguage="$l" 138 | 139 | assertEquals "checking output for $l" \ 140 | "$l" "$(language_getLanguage_SO)" 141 | done 142 | } 143 | 144 | test_setLanguage_fallsbackOnVerificationFailure() { 145 | language_setLanguage_GV "COMPLETE_BOLOCKS" 146 | 147 | assertEquals "checking fallback value" \ 148 | "PL" "$___g_language_napiprojektLanguage" 149 | } 150 | 151 | test_setLanguage_worksForAllSupportedCodes() { 152 | for i in "${!___g_napiprojektLanguageCodes2L[@]}"; do 153 | lang="${___g_napiprojektLanguageCodes2L[$i]}" 154 | language_setLanguage_GV "$lang" 155 | 156 | [ "$lang" = "EN" ] && lang="ENG" 157 | 158 | assertEquals "checking language value" \ 159 | "$lang" "$___g_language_napiprojektLanguage" 160 | done 161 | } 162 | 163 | # shunit call 164 | . shunit2 165 | -------------------------------------------------------------------------------- /actions/libnapi_download.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | ######################################################################## 35 | 36 | # globals 37 | 38 | # 39 | # @brief aggregates all collected subtitle hashes 40 | # 41 | declare -a ___g_subHashes=() 42 | 43 | ___g_downloadMovieTitleFileName=0 44 | 45 | ######################################################################## 46 | 47 | # 48 | # @brief parse the cli arguments 49 | # 50 | _download_argvDispatcher_GV() { 51 | case "$1" in 52 | "-t" | "--title") 53 | ___g_downloadMovieTitleFileName=1 54 | ;; 55 | 56 | "-c" | "--cover") 57 | sysconf_setKey_GV napiprojekt.cover.download 1 58 | ;; 59 | 60 | "-n" | "--nfo") 61 | sysconf_setKey_GV napiprojekt.nfo.download 1 62 | ;; 63 | 64 | "-e" | "--ext") 65 | ___g_argvOutputHandlerType="func" 66 | ___g_argvOutputHandler='sysconf_setKey_GV napiprojekt.subtitles.extension' 67 | ___g_argvErrorMsg=$"nie okreslono domyslnego rozszerzenia dla pobranych plikow" 68 | ;; 69 | 70 | *) 71 | # shellcheck disable=SC2155 72 | local normalizedHash="$(napiprojekt_normalizeHash_SO "$1")" 73 | ___g_subHashes=( "${___g_subHashes[@]}" "$normalizedHash" ) 74 | ;; 75 | esac 76 | } 77 | 78 | _download_parseArgv_GV() { 79 | logging_debug $LINENO $"parsuje opcje akcji download" 80 | argv_argvParser_GV _download_argvDispatcher_GV "$@" 81 | } 82 | 83 | _download_getSubtitlesForHashes() { 84 | local lang="$(language_getLanguage_SO)" 85 | local subsExt=$(subs_getDefaultExtension_SO) 86 | local coverExt=$(sysconf_getKey_SO napiprojekt.cover.extension) 87 | local nfoExt=$(sysconf_getKey_SO napiprojekt.nfo.extension) 88 | 89 | local downloadNfo=$(sysconf_getKey_SO napiprojekt.nfo.download) 90 | local downloadCover=$(sysconf_getKey_SO napiprojekt.cover.download) 91 | 92 | logging_info $LINENO $"pobieram napisy w jezyku" "[$lang]" 93 | 94 | for h in "${___g_subHashes[@]}"; do 95 | local xmlFile="$(fs_mktempFile_SO)" 96 | 97 | if napiprojekt_downloadXml "$h" "" "0" "$xmlFile" "$lang" && 98 | napiprojekt_verifyXml "$xmlFile"; then 99 | local subsFile= 100 | 101 | [ "$___g_downloadMovieTitleFileName" -eq 1 ] && { 102 | logging_debug $LINENO $"ekstrakcja tytulu z XML" 103 | subsFile="$(napiprojekt_extractTitleFromXml_SO "$xmlFile")" 104 | } 105 | 106 | [ -z "$subsFile" ] && { 107 | logging_debug $LINENO $"XML nie zawiera tytulu" 108 | subsFile="${h}" 109 | } 110 | 111 | napiprojekt_extractSubsFromXml "$xmlFile" \ 112 | "${subsFile}.${subsExt}" && 113 | logging_msg $"napisy pobrano pomyslnie" "[$h]" 114 | 115 | [ "$downloadNfo" -eq 1 ] && { 116 | logging_debug $LINENO $"tworze plik nfo" 117 | napiprojekt_extractNfoFromXml "$xmlFile" \ 118 | "${subsFile}.${nfoExt}" && 119 | logging_msg $"plik nfo utworzony pomyslnie" "[$h]" 120 | } 121 | 122 | [ "$downloadCover" -eq 1 ] && { 123 | logging_debug $LINENO $"wypakowuje okladke z XML" 124 | napiprojekt_extractCoverFromXml "$xmlFile" \ 125 | "${subsFile}.${coverExt}" && 126 | logging_msg $"okladka pobrana pomyslnie" "[$h]" 127 | } 128 | fi 129 | done 130 | } 131 | 132 | ######################################################################## 133 | 134 | download_usage() { 135 | echo $"napi.sh download [OPCJE] " 136 | echo 137 | echo $"OPCJE:" 138 | echo $" -c | --cover - pobierz okladke" 139 | echo $" -n | --nfo - utworz plik z informacjami o napisach" 140 | echo $" -t | --title - nazwa pliku jak tytul" 141 | echo $" -e | --ext - rozszerzenie dla pobranych napisow (domyslnie *.txt)" 142 | } 143 | 144 | download_main() { 145 | # parse search specific options 146 | _download_parseArgv_GV "$@" || { 147 | logging_debug $"blad parsera download" 148 | return $G_RETFAIL 149 | } 150 | 151 | # process hashes 152 | _download_getSubtitlesForHashes 153 | return $G_RETOK 154 | } 155 | 156 | # EOF 157 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/assets.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import hashlib 4 | import json 5 | import os 6 | import random 7 | import shutil 8 | import uuid 9 | import logging 10 | 11 | class Assets(object): 12 | """ 13 | Implements simple assets file management 14 | """ 15 | 16 | ASSETS_JSON = 'assets.json' 17 | VERSION = 1 18 | 19 | def __init__(self, path): 20 | """ 21 | The path to multimedia directory. This directory should contain the 22 | assets.json - assets description JSON file. 23 | """ 24 | self.path = path 25 | self.logger = logging.getLogger() 26 | 27 | with open(os.path.join(self.path, self.ASSETS_JSON), "r") as assetsJson: 28 | self.assets = json.load(assetsJson) 29 | 30 | if self.assets['version'] is not self.VERSION: 31 | raise exception.RuntimeError("Unsupported assets version") 32 | 33 | def _prepareMedia(self, sandbox, asset, name): 34 | """ 35 | Copies selected asset file to given sandbox under given name. Asset 36 | descriptor is required as a parameter. 37 | """ 38 | 39 | assetPath = os.path.join(self.path, asset['filename']) 40 | mediaPath = os.path.join(sandbox.path, name) 41 | shutil.copyfile(assetPath, mediaPath) 42 | 43 | self.logger.debug("Preparing asset [{}] -> [{}]".format(assetPath, 44 | mediaPath)) 45 | 46 | # return a media descriptor 47 | return { 'name': name, 'path': mediaPath, 'asset': asset } 48 | 49 | def _mapExtension(self, assetType): 50 | # translates file types to extensions 51 | exts = { 52 | 'mpeg-4': 'mp4' 53 | } 54 | 55 | ext = (assetType if assetType != 'unknown' 56 | else self.DEFAULT_EXT) 57 | try: 58 | extMapped = exts[ext] 59 | ext = extMapped 60 | except KeyError: 61 | pass 62 | 63 | return ext 64 | 65 | def prepareMediaRange(self, sandbox, assetType = None): 66 | """ 67 | Prepares a list of assets of given type 68 | """ 69 | allAssets = (self.assets['assets'] if not assetType else 70 | [ a for a in self.assets['assets'] 71 | if a['type'] == assetType ]) 72 | 73 | preparedAssets = [] 74 | for asset in allAssets: 75 | ext = self._mapExtension(asset['type']) 76 | nameWithSpaces = ' '.join( 77 | (uuid.uuid4().hex, uuid.uuid4().hex)) 78 | name = '.'.join((nameWithSpaces, ext)) 79 | 80 | preparedAssets.append( 81 | self._prepareMedia(sandbox, 82 | asset, 83 | name)) 84 | 85 | return preparedAssets 86 | 87 | def prepareRandomMedia(self, sandbox, 88 | assetType = None, name = None): 89 | """ 90 | Prepares random media file with given name (or generated uuid if name 91 | not given) 92 | """ 93 | allAssets = (self.assets['assets'] if not assetType else 94 | [ a for a in self.assets['assets'] 95 | if a['type'] == assetType ]) 96 | 97 | asset = random.choice(allAssets) 98 | 99 | if not name: 100 | ext = self._mapExtension(asset['type']) 101 | nameWithSpaces = ' '.join( 102 | (uuid.uuid4().hex, uuid.uuid4().hex)) 103 | name = '.'.join((nameWithSpaces, ext)) 104 | 105 | return self._prepareMedia(sandbox, 106 | asset, 107 | name) 108 | 109 | def prepareMedia(self, sandbox, assetId, name): 110 | """ 111 | Prepare media out of specific asset 112 | """ 113 | return self._prepareMedia(sandbox, 114 | self.assets['assets'][assetId], name) 115 | 116 | 117 | class VideoAssets(Assets): 118 | """ 119 | Implements management of multimedia files assets 120 | """ 121 | DEFAULT_EXT = 'avi' 122 | DEFAULT_GENERATED_MEDIA_SIZE = 1024*1024*10 123 | 124 | def _makeFHash(self, md5): 125 | tIdx = [ 0xe, 0x3, 0x6, 0x8, 0x2 ] 126 | tMul = [ 2, 2, 5, 4, 3 ] 127 | tAdd = [ 0, 0xd, 0x10, 0xb, 0x5 ] 128 | digest = "" 129 | 130 | for i in xrange(5): 131 | a = tAdd[i] 132 | m = tMul[i] 133 | g = tIdx[i] 134 | 135 | t = int(md5[g:g+1],16) + a 136 | v = int(md5[t:t+2],16) 137 | 138 | x = (v * m) % 0x10 139 | z = format(x, 'x') 140 | digest = digest + z 141 | return digest 142 | 143 | def generateMedia(self, 144 | sandbox, 145 | size = DEFAULT_GENERATED_MEDIA_SIZE, 146 | name = None): 147 | """ 148 | Prepares a media file with random data inside, with given size. 149 | A fake asset descriptor will be built on the fly for the 150 | generated media file. 151 | """ 152 | 153 | ext = self.DEFAULT_EXT 154 | if not name: 155 | name = '.'.join((uuid.uuid4().hex, ext)) 156 | 157 | mediaPath = os.path.join(sandbox.path, name) 158 | 159 | # prepare fake asset descriptor 160 | asset = { 161 | 'filename': 'dev-urandom.dat', 162 | 'size': size, 163 | 'f': None, 164 | 'md5': None, 165 | 'type': ext, 166 | } 167 | 168 | with open('/dev/urandom', 'rb') as randomDev: 169 | with open(mediaPath, 'wb') as mediaFile: 170 | mediaFile.write(randomDev.read(size)) 171 | 172 | with open(mediaPath, 'rb') as mediaFile: 173 | md5 = hashlib.md5() 174 | md5.update(mediaFile.read()) 175 | asset['md5'] = md5.hexdigest() 176 | asset['f'] = self._makeFHash(asset['md5']) 177 | 178 | return { 'name': name, 'path': mediaPath, 'asset': asset } 179 | 180 | class SubtitlesAssets(Assets): 181 | """ 182 | Implements management of subtitles files assets 183 | """ 184 | DEFAULT_EXT = 'txt' 185 | 186 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # [version 2.0.0 (2017-08-13)](https://github.com/dagon666/napi/releases/tag/v2.0.0) 2 | - Complete functional decomposition into self contained libraries 3 | - Code clean-up, re-design and refactoring 4 | - New self-sustaining test environment with mocked HTTP napiprojekt.pl service 5 | based on docker 6 | - Unit test environment running in docker with code coverage support, shell 7 | functions and shell commands mocks 8 | - Functionality split into actions 9 | - New actions implemented: search, download, subtitles 10 | - Build system replaced with CMake 11 | - Removed most of brittle and clunky custom scripts 12 | 13 | # [version 1.3.5 (2015-06-07)](https://github.com/dagon666/napi/releases/tag/v1.3.5) 14 | - fix for MAC OS having both GNU & BSD stat installed 15 | - install.sh fixes for MAC OS 16 | - configure_cmds function refactoring 17 | - overlap correction implemented for all supported format 18 | - added debian packaging script 19 | 20 | # [version 1.3.4 (2015-01-21)](https://github.com/dagon666/napi/releases/tag/v1.3.4) 21 | - bugfix for system detection routine 22 | - core detection routine bugfix 23 | - test environment corrected test box name 24 | - added -px debugging option to preserve the xml 25 | - various fixes for mac osx 26 | 27 | # [version 1.3.3 (2014-09-11)](https://github.com/dagon666/napi/releases/tag/v1.3.3) 28 | - cleanup on SIGINT implemented 29 | - removed attempts to access /proc on darwin 30 | - corrections for the log being broken 31 | - removed grep -o which is not portable (busybox) 32 | 33 | # [version 1.3.2 (2014-08-08)](https://github.com/dagon666/napi/releases/tag/v1.3.2) 34 | - corrections for ffmpeg framerate detection 35 | - corrections for the logging mechanism 36 | - bugfixes 37 | - subotage correction for minutes increment 38 | 39 | # [version v1.3.1 (2014-07-11)](https://github.com/dagon666/napi/releases/tag/v1.3.1) 40 | - napiprojekt3 XML API support implemented 41 | - napiprojekt3 API as default engine (legacy mode still supported using --id 42 | pynapi or --id other) 43 | - implemented media nfo retrieval from napi xml (napiprojekt3 API) 44 | - implemented cover support using napi xml (napiprojekt3 API) 45 | - fps detection using ffprobe + ffmpeg fps detection fixes 46 | - subotage.sh reimplemented and code cleaned up 47 | - fab support removed from subotage 48 | - napi and subotage code integration 49 | - extraction of common code to the napi_common.sh library 50 | - unit tests for napi and subotage 51 | - system tests for napi and subotage 52 | - a lot of bugfixes and corrections 53 | - napi bundle installer introduced 54 | - many fixes to subotage format processing 55 | - added logoverwrite option to napi 56 | 57 | # [version v1.2.1 (2014-06-11)](https://github.com/dagon666/napi/releases/tag/v1.2.1) 58 | - Major code redesign which includes 59 | -- decomposing code into small functional blocks 60 | -- assuring more compatibility with old shells 61 | -- implemented multithreading (spawning multiple processes at the same time to speed 62 | up the processing -F options) 63 | -- "skip" option reimplemented and made more flexible 64 | -- multiple verbosity levels 65 | - implemented unit test suite for napi.sh 66 | - prepared test environment for system & unit tests (based on Vagrant & Puppet) 67 | - prepared system tests for napi & subotage 68 | 69 | # [version v1.1.13 (2014-05-03)](https://github.com/dagon666/napi/releases/tag/v1.1.13) 70 | - contributions from Maciej Lopacinski (iconv charset conversion) merged 71 | - contributions from github user "emfor" merged (abbreviations support) 72 | - created a test environment 73 | - made napi.sh more compatible with old bash releases 74 | - preparations to write unit tests 75 | 76 | # [version 1.1.12 (2014-04-19)](https://github.com/dagon666/napi/releases/tag/v1.1.12) 77 | - fps detection using ffmpeg added 78 | - corrections to subotage 79 | - Abbreviations support added - you can add any custom string between the filename and it's extension 80 | 81 | # [version 1.1.11 (2014-02-05)](https://github.com/dagon666/napi/releases/tag/v1.1.11) 82 | - napi.sh bugfixes 83 | - added support to download subtitles in a selected language 84 | 85 | # [version 1.1.10 (2014-02-03)](https://github.com/dagon666/napi/releases/tag/v1.1.10) 86 | - napi.sh added a --bigger-than flag, which can be used to download subtitles only for files bigger than the given size (in MB) - that way video samples can be sift out 87 | - various corrections to the file search function 88 | 89 | # [version 1.1.9 (2014-02-03)](https://github.com/dagon666/napi/releases/tag/v1.1.9) 90 | - napi works on Mac OS X, corrections for subotage script - overlap detection function 91 | - napi.sh code cleanup (polish variable names renamed to english) 92 | - merged changes from mcmajkel (Mac OS X compatibility) 93 | - subotage.sh by default subtitles with no end marker will be displayed for 5 seconds 94 | - subotage.sh overlap detection function 95 | 96 | # [version 1.1.8 (2013-11-03)](https://github.com/dagon666/napi/releases/tag/v1.1.8) 97 | - fix for a bug which deleted the resulting subtitles file when the converted file had the same name and extension as the source file 98 | 99 | # [version 1.1.7 (2013-10-26)](https://github.com/dagon666/napi/releases/tag/v1.1.7) 100 | - introduced -e argument to specify default extension for output files. This helps when used in conjunction with subotage the script will detect that the conversion is not needed and will leave the remaining unconverted files with the default extension. 101 | 102 | # [version 1.1.6 (2013-10-25)]() 103 | - subotage and napi will be maintained in a single repository 104 | - github source repository for the project created: https://github.com/dagon666/napi 105 | - contributions from Michal Gorny included 106 | 107 | # [version 1.1.3 (2011-02-04)]() 108 | - Necessary tools presence validation added 109 | - some small bugfixes introduced 110 | 111 | # [version 1.1.2 (2011-01-18)]() 112 | - Logging option was added. Complete operational log can be chosed instead of standard output. 113 | - Some improvements in case that the subtitles file already exists had been introduced - directory sweeping is now performed way faster 114 | 115 | # [version v1.1.1 (2011-01-09)](https://github.com/dagon666/napi/releases/tag/v1.3.5) 116 | - integration with subotage - universal subtitle converter. napi.sh is now able to convert subtitles to one of the supported destination formats on the fly - right after the original file has been downloaded 117 | - a lot of changes in the processing functions 118 | - some small bugfixes related with argument processing 119 | - user authorisation data can now be passed as arguments 120 | 121 | # [version v0.1.8 (2010-06-04)](https://github.com/dagon666/napi/releases/tag/v1.3.5) 122 | - Added support to download covers 123 | - 7zip is no longer obligatory !!! 124 | - code cleanup 125 | 126 | -------------------------------------------------------------------------------- /tests/integration_tests/test_languages.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import re 4 | 5 | import napi.fs 6 | import napi.sandbox 7 | import napi.subtitles 8 | import napi.testcase 9 | 10 | class LanguagesTest(napi.testcase.NapiTestCase): 11 | 12 | def _makeLanguageDebugRegex(self, lang): 13 | return re.compile(r'jezyk skonfigurowany jako [{}]'.format(lang)) 14 | 15 | def test_ifDownloadsSubtitlesInDifferentLanguages(self): 16 | """ 17 | Brief: 18 | Check if downloads subtitles in different languages without 19 | errors. 20 | 21 | Procedure: 22 | 1. Prepare a media file 23 | 2. Program napiprojekt mock 24 | 3. Call napi with -L ENG 25 | 26 | Expected Results: 27 | Check if downloads subtitles file and doesn't generate errors on 28 | output. 29 | """ 30 | media = None 31 | lang = 'ENG' 32 | 33 | with napi.sandbox.Sandbox() as sandbox: 34 | media = self.videoAssets.prepareRandomMedia(sandbox) 35 | 36 | # program http mock 37 | self.napiMock.programXmlRequest( 38 | media, 39 | napi.subtitles.CompressedSubtitles.fromString( 40 | media['asset'], "test subtitles")) 41 | 42 | self.napiScan('--stats', '-L', lang, media['path']) 43 | self.output.stdoutContains(self._makeLanguageDebugRegex(lang)) 44 | 45 | stats = self.output.parseNapiStats() 46 | 47 | self.assertEquals(1, stats['ok']) 48 | self.assertEquals(1, stats['total']) 49 | self.assertEquals(0, stats['conv_charset']) 50 | self.assertEquals(0, stats['unav']) 51 | 52 | self.assertTrue( 53 | napi.fs.Filesystem(media).subtitlesExists()) 54 | 55 | def test_ifUsesProvidedExtension(self): 56 | """ 57 | Brief: 58 | Request subtitles in different language and specify an extension 59 | for the resulting file. 60 | 61 | Procedure: 62 | 1. Prepare a media file 63 | 2. Program napiprojekt mock 64 | 3. Call napi with -L ENG -e eng 65 | 66 | Expected Results: 67 | Should download a file successfully and its resulting name should 68 | be ended with provided extension. 69 | """ 70 | media = None 71 | lang = 'ENG' 72 | ext = lang.lower() 73 | 74 | with napi.sandbox.Sandbox() as sandbox: 75 | media = self.videoAssets.prepareRandomMedia(sandbox) 76 | 77 | # program http mock 78 | self.napiMock.programXmlRequest( 79 | media, 80 | napi.subtitles.CompressedSubtitles.fromString( 81 | media['asset'], "test subtitles")) 82 | 83 | self.napiScan('--stats', '-L', lang, '-e', ext, media['path']) 84 | self.output.stdoutContains(self._makeLanguageDebugRegex(lang)) 85 | 86 | stats = self.output.parseNapiStats() 87 | 88 | self.assertEquals(1, stats['ok']) 89 | self.assertEquals(1, stats['total']) 90 | self.assertEquals(0, stats['conv_charset']) 91 | self.assertEquals(0, stats['unav']) 92 | 93 | self.assertTrue( 94 | napi.fs.Filesystem(media).subtitlesExists(None, ext)) 95 | 96 | def test_ifUsesProvidedAbbreviation(self): 97 | """ 98 | Brief: 99 | Request subtitles in different language and specify an 100 | abbreviation for the resulting file. 101 | 102 | Procedure: 103 | 1. Prepare a media file 104 | 2. Program napiprojekt mock 105 | 3. Call napi with -L ENG -a eng 106 | 107 | Expected Results: 108 | Should download a file successfully and its resulting name should 109 | contain the string provided as abbreviation, placed in between 110 | the file name and the extension. 111 | """ 112 | media = None 113 | lang = 'ENG' 114 | abbrev = lang.lower() 115 | 116 | with napi.sandbox.Sandbox() as sandbox: 117 | media = self.videoAssets.prepareRandomMedia(sandbox) 118 | 119 | # program http mock 120 | self.napiMock.programXmlRequest( 121 | media, 122 | napi.subtitles.CompressedSubtitles.fromString( 123 | media['asset'], "test subtitles")) 124 | 125 | self.napiScan('--stats', '-L', lang, '-a', abbrev, media['path']) 126 | self.output.stdoutContains(self._makeLanguageDebugRegex(lang)) 127 | 128 | stats = self.output.parseNapiStats() 129 | 130 | self.assertEquals(1, stats['ok']) 131 | self.assertEquals(1, stats['total']) 132 | self.assertEquals(0, stats['conv_charset']) 133 | self.assertEquals(0, stats['unav']) 134 | 135 | self.assertTrue( 136 | napi.fs.Filesystem(media).subtitlesExists(None, None, abbrev)) 137 | 138 | def test_ifUsesProvidedAbbreviationAndExtension(self): 139 | """ 140 | Brief: 141 | Request subtitles in different language and specify an 142 | abbreviation and the extension for the resulting file. 143 | 144 | Procedure: 145 | 1. Prepare a media file 146 | 2. Program napiprojekt mock 147 | 3. Call napi with -L ENG -a eng -e test-extension 148 | 149 | Expected Results: 150 | Should download a file successfully and its resulting name should 151 | contain the string provided as abbreviation, placed in between 152 | the file name and the provided extension. 153 | """ 154 | media = None 155 | lang = 'ENG' 156 | abbrev = lang.lower() 157 | ext = "test-extension" 158 | 159 | with napi.sandbox.Sandbox() as sandbox: 160 | media = self.videoAssets.prepareRandomMedia(sandbox) 161 | 162 | # program http mock 163 | self.napiMock.programXmlRequest( 164 | media, 165 | napi.subtitles.CompressedSubtitles.fromString( 166 | media['asset'], "test subtitles")) 167 | 168 | self.napiScan('--stats', '-L', lang, 169 | '-a', abbrev, 170 | '-e', ext, 171 | media['path']) 172 | 173 | self.output.stdoutContains(self._makeLanguageDebugRegex(lang)) 174 | 175 | stats = self.output.parseNapiStats() 176 | 177 | self.assertEquals(1, stats['ok']) 178 | self.assertEquals(1, stats['total']) 179 | self.assertEquals(0, stats['conv_charset']) 180 | self.assertEquals(0, stats['unav']) 181 | 182 | self.assertTrue( 183 | napi.fs.Filesystem(media).subtitlesExists(None, ext, abbrev)) 184 | 185 | if __name__ == '__main__': 186 | napi.testcase.runTests() 187 | -------------------------------------------------------------------------------- /tests/unit_tests/libnapi_subs_test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # module dependencies 35 | . ../../libs/libnapi_assoc.sh 36 | . ../../libs/libnapi_retvals.sh 37 | . ../../libs/libnapi_sysconf.sh 38 | . ../../libs/libnapi_wrappers.sh 39 | 40 | # fakes/mocks 41 | . fake/libnapi_logging_fake.sh 42 | . mock/scpmocker.sh 43 | 44 | # module under test 45 | . ../../libs/libnapi_subs.sh 46 | 47 | # 48 | # tests env setup 49 | # 50 | setUp() { 51 | 52 | # restore original values 53 | ___g_subs_defaultExtension='txt' 54 | sysconf_setKey_GV napiprojekt.subtitles.extension 'txt' 55 | scpmocker_setUp 56 | } 57 | 58 | # 59 | # tests env tear down 60 | # 61 | tearDown() { 62 | scpmocker_tearDown 63 | } 64 | 65 | test_subs_getSubFormatExtension_SO_returnsExpectedExtensions() { 66 | local formats=( 'subrip' 'subviewer2' 'other' \ 67 | 'non-existing-format' ) 68 | local defaultExt="$(sysconf_getKey_SO napiprojekt.subtitles.extension)" 69 | local expected=( 'srt' 'sub' \ 70 | "$defaultExt" "$defaultExt" ) 71 | 72 | local idx= 73 | for idx in "${!formats[@]}"; do 74 | local f="${formats[$idx]}" 75 | local e="${expected[$idx]}" 76 | 77 | assertEquals "checking extension for format [$f]" \ 78 | "$e" "$(subs_getSubFormatExtension_SO "$f")" 79 | done 80 | } 81 | 82 | test_subs_getDefaultExtension_SO_returnsTheGlobalVariableValue() { 83 | local value= 84 | for value in "{a..z}{a..z}"; do 85 | sysconf_setKey_GV napiprojekt.subtitles.extension "$value" 86 | assertEquals "check return value" \ 87 | "$value" "$(subs_getDefaultExtension_SO)" 88 | done 89 | } 90 | 91 | test_subs_getCharset_failsIfFileToolIsNotDetected() { 92 | local subsFile=$(mktemp -p "${SHUNIT_TMPDIR}") 93 | 94 | scpmocker_patchFunction tools_isDetected 95 | scpmocker -c func_tools_isDetected program -e 1 96 | 97 | subs_getCharset_SO "$subsFile" 98 | 99 | assertEquals "check exit value" \ 100 | "$G_RETFAIL" "$?" 101 | 102 | scpmocker_resetFunction tools_isDetected 103 | } 104 | 105 | test_subs_getCharset_detectsAllSupportedCharsets() { 106 | local subsFile=$(mktemp -p "${SHUNIT_TMPDIR}") 107 | 108 | scpmocker_patchCommand file 109 | scpmocker_patchFunction tools_isDetected 110 | 111 | scpmocker -c func_tools_isDetected program -e 0 112 | scpmocker -c file program -s "utf-8" 113 | scpmocker -c file program -s "utf8" 114 | scpmocker -c file program -s "iso-885" 115 | scpmocker -c file program -s "us-ascii" 116 | scpmocker -c file program -s "csascii" 117 | scpmocker -c file program -s "ascii" 118 | scpmocker -c file program -s "other" 119 | 120 | assertEquals "check output for charset utf" \ 121 | "UTF8" "$(subs_getCharset_SO "$subsFile")" 122 | 123 | assertEquals "check output for charset utf" \ 124 | "UTF8" "$(subs_getCharset_SO "$subsFile")" 125 | 126 | assertEquals "check output for charset iso" \ 127 | "ISO-8859-2" "$(subs_getCharset_SO "$subsFile")" 128 | 129 | assertEquals "check output for charset usascii" \ 130 | "US-ASCII" "$(subs_getCharset_SO "$subsFile")" 131 | 132 | assertEquals "check output for charset csascii" \ 133 | "CSASCII" "$(subs_getCharset_SO "$subsFile")" 134 | 135 | assertEquals "check output for charset ascii" \ 136 | "ASCII" "$(subs_getCharset_SO "$subsFile")" 137 | 138 | assertEquals "check output for charset ascii" \ 139 | "WINDOWS-1250" "$(subs_getCharset_SO "$subsFile")" 140 | 141 | scpmocker_resetFunction tools_isDetected 142 | } 143 | 144 | test_subs_convertEncodingDoesntOverwriteOriginalIfConversionFails() { 145 | local subsFile=$(mktemp -p "${SHUNIT_TMPDIR}") 146 | local convFile=$(mktemp -p "${SHUNIT_TMPDIR}") 147 | local testData="Mary had a little lamb" 148 | 149 | scpmocker_patchCommand iconv 150 | scpmocker_patchFunction subs_getCharset_SO 151 | scpmocker_patchFunction fs_mktempFile_SO 152 | 153 | # iconv failure 154 | scpmocker -c iconv program -e 1 155 | scpmocker -c func_subs_getCharset_SO program -s "UTF8" 156 | scpmocker -c func_fs_mktempFile_SO program -s "$convFile" 157 | 158 | echo "$testData" > "$subsFile" 159 | local expectedHash=$(sha256sum "$subsFile" | awk '{ print $1 }') 160 | 161 | subs_convertEncoding "$subsfile" "some encoding" 162 | 163 | local dataHash=$(sha256sum "$subsFile" | awk '{ print $1 }') 164 | 165 | assertEquals "compare file hashes" \ 166 | "$expectedHash" "$dataHash" 167 | 168 | assertEquals "check iconv's mock call count" \ 169 | "1" "$(scpmocker -c iconv status -C)" 170 | 171 | scpmocker_resetFunction subs_getCharset_SO 172 | scpmocker_resetFunction fs_mktempFile_SO 173 | } 174 | 175 | test_subs_convertEncodingReplacesTheFileIfConversionSuccessful() { 176 | local subsFile=$(mktemp -p "${SHUNIT_TMPDIR}") 177 | local convFile=$(mktemp -p "${SHUNIT_TMPDIR}") 178 | 179 | scpmocker_patchCommand iconv 180 | scpmocker_patchCommand mv 181 | scpmocker_patchFunction subs_getCharset_SO 182 | scpmocker_patchFunction fs_mktempFile_SO 183 | 184 | # iconv failure 185 | scpmocker -c iconv program -e 0 186 | scpmocker -c func_subs_getCharset_SO program -s "UTF8" 187 | scpmocker -c func_fs_mktempFile_SO program -s "$convFile" 188 | 189 | subs_convertEncoding "$subsfile" "some encoding" 190 | 191 | assertEquals "check iconv's mock call count" \ 192 | "1" "$(scpmocker -c iconv status -C)" 193 | 194 | assertEquals "check mv's mock call count" \ 195 | "1" "$(scpmocker -c mv status -C)" 196 | 197 | scpmocker_resetFunction subs_getCharset_SO 198 | scpmocker_resetFunction fs_mktempFile_SO 199 | } 200 | 201 | # shunit call 202 | . shunit2 203 | -------------------------------------------------------------------------------- /tests/unit_tests/libnapi_http_test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # module dependencies 35 | . ../../libs/libnapi_retvals.sh 36 | 37 | 38 | # fakes/mocks 39 | . fake/libnapi_logging_fake.sh 40 | . mock/scpmocker.sh 41 | 42 | # module under test 43 | . ../../libs/libnapi_http.sh 44 | 45 | setUp() { 46 | scpmocker_setUp 47 | 48 | # restore original values 49 | ___g_wget='none' 50 | } 51 | 52 | tearDown() { 53 | scpmocker_tearDown 54 | } 55 | 56 | test_http_configure_fallsbackToDefaultWgetInvocation() { 57 | scpmocker_patchCommand "wget" 58 | 59 | # no print status support 60 | scpmocker -c wget program 61 | 62 | # no post support 63 | scpmocker -c wget program -e 1 64 | 65 | http_configure_GV 66 | 67 | assertEquals "check wget command" \ 68 | "0|wget -q -O" "$___g_wget" 69 | 70 | assertEquals "check first mock invocation" \ 71 | "--help" "$(scpmocker -c wget status -A 1)" 72 | 73 | assertEquals "check second mock invocation" \ 74 | "--help" "$(scpmocker -c wget status -A 2)" 75 | } 76 | 77 | test_http_configure_detectsServerResponseSupport() { 78 | scpmocker_patchCommand "wget" 79 | 80 | # print status support 81 | scpmocker -c wget program -s \ 82 | "-S, --server-response print server response." 83 | 84 | # no post support 85 | scpmocker -c wget program -e 1 86 | 87 | http_configure_GV 88 | 89 | assertEquals "check wget command" \ 90 | "0|wget -q -S -O" "$___g_wget" 91 | 92 | assertEquals "check first mock invocation" \ 93 | "--help" "$(scpmocker -c wget status -A 1)" 94 | 95 | assertEquals "check second mock invocation" \ 96 | "--help" "$(scpmocker -c wget status -A 2)" 97 | } 98 | 99 | test_http_configure_detectsPostSupport() { 100 | scpmocker_patchCommand "wget" 101 | 102 | # print status support 103 | scpmocker -c wget program -s \ 104 | "-S, --server-response print server response." 105 | 106 | # no post support 107 | scpmocker -c wget program -e 0 -s \ 108 | "--post-data=STRING use the POST method; send STRING as the data. \ 109 | --post-file=FILE use the POST method; send contents of FILE." 110 | 111 | http_configure_GV 112 | 113 | assertEquals "check wget command" \ 114 | "1|wget -q -S -O" "$___g_wget" 115 | 116 | assertEquals "check first mock invocation" \ 117 | "--help" "$(scpmocker -c wget status -A 1)" 118 | 119 | assertEquals "check second mock invocation" \ 120 | "--help" "$(scpmocker -c wget status -A 2)" 121 | } 122 | 123 | test_http_wget_callsConfiguredCommand() { 124 | scpmocker_patchCommand "wget" 125 | 126 | ___g_wget="0|wget" 127 | http_wget 128 | 129 | ___g_wget="123|wget" 130 | http_wget 131 | 132 | ___g_wget="wget" 133 | http_wget 134 | 135 | ___g_wget="1111|wget" 136 | http_wget 137 | 138 | assertEquals "check mock call count" \ 139 | 4 "$(scpmocker -c wget status -C)" 140 | } 141 | 142 | test_http_isPostRequestSupported_detection() { 143 | ___g_wget="1|wget" 144 | assertTrue "check post support" \ 145 | http_isPostRequestSupported 146 | 147 | ___g_wget="0|wget" 148 | assertFalse "check no post support" \ 149 | http_isPostRequestSupported 150 | 151 | ___g_wget="123|wget" 152 | assertFalse "check no post support 2" \ 153 | http_isPostRequestSupported 154 | } 155 | 156 | test_http_getHttpStatusExtractsStatusFromWgetOutput() { 157 | local wgetOutput= 158 | 159 | read -d "" wgetOutput << EOF 160 | HTTP/1.1 302 Found 161 | Cache-Control: private 162 | Content-Type: text/html; charset=UTF-8 163 | Location: http://www.google.co.uk/?gfe_rd=cr&ei=jjOnWNWyAcjU8geznrugCQ 164 | Content-Length: 261 165 | Date: Fri, 17 Feb 2017 17:31:58 GMT 166 | HTTP/1.1 200 OK 167 | Date: Fri, 17 Feb 2017 17:31:58 GMT 168 | Expires: -1 169 | Cache-Control: private, max-age=0 170 | Content-Type: text/html; charset=ISO-8859-1 171 | P3P: CP="This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info." 172 | Server: gws 173 | X-XSS-Protection: 1; mode=block 174 | X-Frame-Options: SAMEORIGIN 175 | Set-Cookie: NID=97=OkKeQzJpTOAOi5Z2Y4ARk-4s1au-Er-Jas1ym-lfunnKUv2P4G6taMrJAeJQX3dq68IHTp3wxmCr_w_26v9ykdYQt6rllmCdvUXVo4deK2vU3hTDy8ThulGLmbw5dM9o; expires=Sat, 19-Aug-2017 17:31:58 GMT; path=/; domain=.google.co.uk; HttpOnly 176 | Accept-Ranges: none 177 | Vary: Accept-Encoding 178 | Transfer-Encoding: chunked 179 | EOF 180 | 181 | assertEquals "check command output" \ 182 | "302 200" "$(echo "$wgetOutput" | http_getHttpStatus)" 183 | } 184 | 185 | test_http_downloadUrl_outputsDataToStdoutAndStatusToStderr() { 186 | local fakeData="some fake response data" 187 | 188 | scpmocker_patchCommand "wget" 189 | scpmocker -c wget program -s "$fakeData" 190 | 191 | ___g_wget="1|wget -q -S -O" 192 | 193 | local data=$(http_downloadUrl_SOSE \ 194 | "http://some.fake.url") 195 | 196 | assertEquals "check command output" \ 197 | "$fakeData" "$data" 198 | 199 | assertEquals "check mock call count" \ 200 | 1 "$(scpmocker -c wget status -C)" 201 | } 202 | 203 | test_http_downloadUrl_handlesPostRequests() { 204 | local fakeData="some fake response data" 205 | local postData="some post data" 206 | local fakeUrl="http://some.fake.url" 207 | 208 | scpmocker_patchCommand "wget" 209 | scpmocker -c wget program -s "$fakeData" 210 | 211 | ___g_wget="1|wget -q -S -O" 212 | 213 | local data=$(http_downloadUrl_SOSE \ 214 | "$fakeUrl" "" "$postData") 215 | 216 | assertEquals "checking mock positionals" \ 217 | "-q -S -O /dev/stdout --post-data=${postData} $fakeUrl" \ 218 | "$(scpmocker -c wget status -A 1)" 219 | } 220 | 221 | 222 | # shunit call 223 | . shunit2 224 | -------------------------------------------------------------------------------- /libs/libnapi_assoc.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # 35 | # @brief generic group lookup function 36 | # @param group name 37 | # @param extract function 38 | # @param array 39 | # 40 | _assoc_groupLookupGeneric_SO() { 41 | local i='' 42 | local results='' 43 | local group="${1}" && shift 44 | local extractor="${1}" && shift 45 | 46 | for i in "$@"; do 47 | local tg= 48 | tg=$(assoc_getGroup_SO "$i") 49 | if [ -n "$tg" ] && [ "$tg" = "$group" ]; then 50 | local tk= 51 | tk=$("$extractor" "$i") 52 | 53 | [ -z "$results" ] && 54 | results="$tk" || 55 | results="$results $tk" 56 | fi 57 | done 58 | 59 | [ -n "$results" ] && echo "$results" 60 | } 61 | 62 | # 63 | # @brief generic key lookup function 64 | # @param key name 65 | # @param extractor function 66 | # @param array 67 | # 68 | _assoc_lookupKeyGeneric_SO() { 69 | local key="$1" && shift 70 | local extractor="$1" && shift 71 | local i='' 72 | local tk='' 73 | 74 | for i in "$@"; do 75 | tk="$(assoc_getKey_SO "$i")" 76 | if [ "$tk" = "$key" ]; then 77 | "$extractor" "$i" 78 | return $G_RETOK 79 | fi 80 | done 81 | return $G_RETFAIL 82 | } 83 | 84 | ######################################################################## 85 | 86 | # 87 | # @brief get the group from strings like group:key=value or key=value 88 | # 89 | assoc_getGroup_SO() { 90 | local k="${1%=*}" 91 | local g="${k%%:*}" 92 | [ "$k" != "$g" ] && echo "$g" 93 | } 94 | 95 | # 96 | # @brief get the value from strings like group:key=value or key=value 97 | # 98 | assoc_getValue_SO() { 99 | echo "${1#*=}" 100 | } 101 | 102 | # 103 | # @brief get the key from strings like group:key=value or key=value 104 | # 105 | assoc_getKey_SO() { 106 | local k="${1%%=*}" 107 | echo "${k#*:}" 108 | } 109 | 110 | # 111 | # @brief search for specified key and return it's value 112 | # @param key 113 | # @param array 114 | # 115 | assoc_lookupValue_SO() { 116 | local key="$1" && shift 117 | _assoc_lookupKeyGeneric_SO "$key" "assoc_getValue_SO" "$@" 118 | } 119 | 120 | # 121 | # @brief modify value in the array (it will be added if the key doesn't exist) 122 | # 123 | # WARNING: Due to specifics of bash it's impossible to return an 124 | # array with containing white characters in elements thus this 125 | # function only works if your groups,keys and values don't contain 126 | # any spaces 127 | # 128 | # @param key 129 | # @param value 130 | # @param array 131 | # 132 | assoc_modifyValue_SO() { 133 | local key="$1" && shift 134 | local value="$1" && shift 135 | 136 | local i=0 137 | local k= 138 | local g= 139 | local rv=() 140 | 141 | 142 | # shellcheck disable=SC2048 143 | for i in $*; do 144 | k=$(assoc_getKey_SO "$i") 145 | 146 | # unfortunately old shells don't support rv+=( "$i" ) 147 | [ "$k" != "$key" ] && rv=( "${rv[@]}" "$i" ) 148 | [ "$k" = "$key" ] && g=$(assoc_getGroup_SO "$i") 149 | done 150 | 151 | if [ -n "$g" ]; then 152 | rv=( "${rv[@]}" "${g}:${key}=${value}" ) 153 | else 154 | rv=( "${rv[@]}" "${key}=${value}" ) 155 | fi 156 | echo ${rv[*]} 157 | } 158 | 159 | # 160 | # @brief modify value in the array (it will be added if the key doesn't exist) 161 | # 162 | # This function expects global variable name as parameter. It will 163 | # modify the array in place. 164 | # 165 | # @param key 166 | # @param value 167 | # @param array name 168 | # 169 | assoc_modifyValue_GV() { 170 | local key="$1" 171 | local value="$2" 172 | local arrayName="$3" 173 | 174 | local i=0 175 | local k= 176 | local g= 177 | local tmp=() 178 | local rv=() 179 | 180 | # copy global to local 181 | eval tmp=\( \"\${${arrayName}[@]}\" \) 182 | 183 | for i in "${tmp[@]}"; do 184 | k=$(assoc_getKey_SO "$i") 185 | 186 | # unfortunately old shells don't support rv+=( "$i" ) 187 | [ "$k" != "$key" ] && rv=( "${rv[@]}" "$i" ) 188 | [ "$k" = "$key" ] && g=$(assoc_getGroup_SO "$i") 189 | done 190 | 191 | if [ -n "$g" ]; then 192 | rv=( "${rv[@]}" "${g}:${key}=${value}" ) 193 | else 194 | rv=( "${rv[@]}" "${key}=${value}" ) 195 | fi 196 | 197 | # re-assign 198 | eval "$arrayName"=\( \"\${rv[@]}\" \) 199 | } 200 | 201 | # 202 | # @brief lookup index in the array for given key 203 | # returns the index of the value and 0 on success, 204 | # @param key 205 | # @param array 206 | # 207 | assoc_lookupKey_SO() { 208 | local i='' 209 | local idx=0 210 | local key="$1" 211 | local tk= 212 | 213 | shift 214 | for i in "$@"; do 215 | tk="$(assoc_getKey_SO "$i")" 216 | [ "$tk" = "$key" ] && { 217 | echo "$idx" 218 | return $G_RETOK 219 | } 220 | idx=$(( idx + 1 )) 221 | done 222 | 223 | echo "-1" 224 | return $G_RETFAIL 225 | } 226 | 227 | # 228 | # @brief search for specified group and return it's keys as string 229 | # @param group 230 | # @param array 231 | # 232 | assoc_lookupGroupKeys_SO() { 233 | local group="${1}" && shift 234 | _assoc_groupLookupGeneric_SO "$group" "assoc_getKey_SO" "$@" 235 | } 236 | 237 | # 238 | # @brief given a key extract it's group (if it has any) 239 | # @param key 240 | # @param array 241 | # 242 | assoc_lookupKeysGroup_SO() { 243 | local group="${1}" && shift 244 | _assoc_lookupKeyGeneric_SO "$group" "assoc_getGroup_SO" "$@" 245 | } 246 | 247 | # 248 | # @brief extract a key=value from an entry of form [group:]key=value 249 | # 250 | assoc_getKvPair_SO() { 251 | echo "${1#*:}" 252 | } 253 | 254 | # 255 | # @brief search for specified group and return it's keys=value pairs as string 256 | # @param group 257 | # @param array 258 | # 259 | assoc_lookupGroupKv_SO() { 260 | local group="${1}" && shift 261 | _assoc_groupLookupGeneric_SO "$group" "assoc_getKvPair_SO" "$@" 262 | } 263 | 264 | # EOF 265 | -------------------------------------------------------------------------------- /libs/libnapi_subs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | ######################################################################## 35 | 36 | # 37 | # @brief convert charset of the file 38 | # @param input file path 39 | # @param output charset 40 | # @param input charset or null 41 | # 42 | _subs_convertEncoding() { 43 | local filePath="$1" 44 | local destEncoding="${2:-utf8}" 45 | local sourceEncoding="${3:-}" 46 | 47 | # detect charset 48 | [ -z "$sourceEncoding" ] && 49 | sourceEncoding=$(subs_getCharset_SO "$filePath") 50 | 51 | local tmp="$(fs_mktempFile_SO)" 52 | iconv -f "$sourceEncoding" -t "$destEncoding" \ 53 | "$filePath" > "$tmp" 2>/dev/null && { 54 | logging_debug $LINENO $"konwersja kodowania pomyslna, zamieniam pliki" 55 | mv "$tmp" "$filePath" 56 | } 57 | } 58 | 59 | ######################################################################## 60 | 61 | # 62 | # @brief get extension for given subtitle format 63 | # 64 | subs_getSubFormatExtension_SO() { 65 | declare -a fmte=( 'subrip=srt' 'subviewer2=sub' ) 66 | assoc_lookupValue_SO "$1" "${fmte[@]}" || subs_getDefaultExtension_SO 67 | } 68 | 69 | # 70 | # @brief echoes default subtitles extensions 71 | # 72 | subs_getDefaultExtension_SO() { 73 | sysconf_getKey_SO napiprojekt.subtitles.extension 74 | } 75 | 76 | # 77 | # @brief detects the charset of the subtitles file 78 | # @param full path to the subtitles file 79 | # 80 | subs_getCharset_SO() { 81 | local file="$1" 82 | local charset= 83 | local et= 84 | 85 | tools_isDetected "file" || return $G_RETFAIL 86 | 87 | et=$(file \ 88 | --brief \ 89 | --mime-encoding \ 90 | --exclude apptype \ 91 | --exclude tokens \ 92 | --exclude cdf \ 93 | --exclude compress \ 94 | --exclude elf \ 95 | --exclude soft \ 96 | --exclude tar \ 97 | "$file" | wrappers_lcase_SO) || { 98 | return $G_RETFAIL 99 | } 100 | 101 | case "$et" in 102 | *utf*) charset="UTF8";; 103 | *iso*) charset="ISO-8859-2";; 104 | us-ascii) charset="US-ASCII";; 105 | csascii) charset="CSASCII";; 106 | *ascii*) charset="ASCII";; 107 | *) charset="WINDOWS-1250";; 108 | esac 109 | 110 | echo "$charset" 111 | } 112 | 113 | # 114 | # @brief convert charset of the file 115 | # @param input file path 116 | # @param encoding 117 | # 118 | subs_convertEncoding() { 119 | local filePath="$1" 120 | local encoding="${2:-}" 121 | local fileName=$(basename "$filePath") 122 | 123 | logging_msg "[$fileName]" $"konwertowanie kodowania do" "$encoding" 124 | _subs_convertEncoding "$filePath" "$encoding" 125 | } 126 | 127 | # 128 | # @brief convert format 129 | # @param full path to the media file 130 | # @param full path of the media file (without filename) 131 | # @param original (as downloaded) subtitles filename 132 | # @param filename to which unconverted subtitles should be renamed 133 | # @param filename for converted subtitles 134 | # @param requested subtitles format 135 | # 136 | subs_convertFormat() { 137 | local videoFilePath="$1" 138 | local videoFileDir="$2" 139 | local sourceSubsFileName="$3" 140 | local originalFileName="$4" 141 | local destSubsFileName="$5" 142 | local format="$6" 143 | 144 | local isDeleteOrigSet="$(sysconf_getKey_SO \ 145 | napiprojekt.subtitles.orig.delete)" 146 | 147 | # verify original file existence before proceeding further 148 | # shellcheck disable=SC2086 149 | [ -e "${videoFileDir}/${sourceSubsFileName}" ] || { 150 | logging_error $"oryginalny plik nie istnieje" 151 | return $G_RETFAIL 152 | } 153 | 154 | # for the backup 155 | local tmp="$(fs_mktempFile_SO)" 156 | 157 | # create backup 158 | logging_debug $LINENO $"backupuje oryginalny plik jako" "$tmp" 159 | cp "${videoFileDir}/${sourceSubsFileName}" "$tmp" 160 | 161 | if [ "$isDeleteOrigSet" -eq 1 ]; then 162 | fs_garbageCollect "${videoFileDir}/${originalFileName}" 163 | else 164 | logging_info $LINENO $"kopiuje oryginalny plik jako" \ 165 | "[$originalFileName]" 166 | 167 | logging_debug $LINENO "source: ${sourceSubsFileName}," \ 168 | "orig: ${originalFileName}" 169 | 170 | [ "${sourceSubsFileName}" != "${originalFileName}" ] && 171 | cp "${videoFileDir}/${sourceSubsFileName}" \ 172 | "${videoFileDir}/${originalFileName}" 173 | fi 174 | 175 | # detect video file framerate 176 | local fps= 177 | 178 | fps=$(fs_getFps_SO "$videoFilePath") 179 | if [ -n "$fps" ] && [ "$fps" != "0" ]; then 180 | logging_msg $"wykryty fps" "$fps" 181 | else 182 | logging_msg $"fps nieznany, okr. na podstawie napisow albo wart. domyslna" 183 | fps=0 184 | fi 185 | 186 | # attempt conversion 187 | local convStatus= 188 | logging_msg $"wolam subotage" 189 | subotage_processFile \ 190 | "${videoFileDir}/${sourceSubsFileName}" \ 191 | "none" \ 192 | "0" \ 193 | "" \ 194 | "${videoFileDir}/${destSubsFileName}" \ 195 | "${format}" \ 196 | "${fps}" \ 197 | "" 198 | convStatus=$? 199 | 200 | if [ "$convStatus" -eq "$G_RETOK" ]; then 201 | # remove the old format if conversion was successful 202 | logging_msg $"pomyslnie przekonwertowano do" "$format" 203 | 204 | [ "$sourceSubsFileName" != "$destSubsFileName" ] && { 205 | logging_info $LINENO "usuwam oryginalny plik" 206 | fs_garbageCollect "${videoFileDir}/${sourceSubsFileName}" 207 | } 208 | 209 | elif [ "$convStatus" -eq $G_RETNOACT ]; then 210 | logging_msg $"subotage - konwersja nie jest konieczna" 211 | 212 | # copy the backup to converted 213 | cp "$tmp" "${videoFileDir}/${destSubsFileName}" 214 | 215 | # get rid of the original file 216 | [ "$sourceSubsFileName" != "$destSubsFileName" ] && 217 | logging_msg "usuwam oryginalny plik" && 218 | fs_unlink "${videoFileDir}/${sourceSubsFileName}" 219 | 220 | else 221 | logging_msg $"konwersja do" "$format" $"niepomyslna" 222 | # restore the backup (the original file may be corrupted due to failed conversion) 223 | cp "$tmp" "$videoFileDir/$sourceSubsFileName" 224 | return $G_RETFAIL 225 | fi 226 | 227 | return $G_RETOK 228 | } 229 | 230 | # EOF 231 | -------------------------------------------------------------------------------- /libs/libnapi_tools.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # 35 | # @brief global tools array 36 | # =1 - mandatory tool 37 | # =0 - optional tool 38 | # 39 | # Syntax: [group:]= 40 | # 41 | declare -a ___g_tools=( 'tr=1' 'printf=1' 'mktemp=1' 'wget=1' \ 42 | 'wc=1' 'dd=1' 'grep=1' 'seq=1' 'sed=1' 'cut=1' \ 43 | 'base64=1' 'unlink=0' 'stat=1' 'basename=1' \ 44 | 'dirname=1' 'cat=1' 'cp=1' 'mv=1' 'awk=1' \ 45 | 'file=0' 'iconv=0' '7z|7za=0' 'md5|md5sum=1' \ 46 | 'fps:mediainfo=0' 'fps:mplayer|mplayer2=0' 'fps:ffmpeg|ffprobe=0' ) 47 | 48 | ######################################################################## 49 | 50 | # 51 | # @brief verify all the registered tools from the tools array 52 | # 53 | tools_configure_GV() { 54 | # this function can cope with that kind of input 55 | # shellcheck disable=SC2068 56 | ___g_tools=( $(tools_verify_SO "${___g_tools[@]}") ) 57 | } 58 | 59 | # 60 | # @brief check function presence 61 | # @param function name 62 | # 63 | tools_verifyFunctionPresence() { 64 | local tool=$(builtin type -t "$1") 65 | 66 | # make sure it's really there 67 | if [ -z "$tool" ]; then 68 | type "$1" >/dev/null 2>&1 || tool='empty' 69 | fi 70 | 71 | # check the output 72 | [ "$tool" = "function" ] 73 | } 74 | 75 | # 76 | # @brief checks if the tool is available in the PATH 77 | # 78 | tools_verifyToolPresence() { 79 | local tool=$(builtin type -p "$1") 80 | local rv=$G_RETUNAV 81 | 82 | # make sure it's really there 83 | if [ -z "$tool" ]; then 84 | type "$1" >/dev/null 2>&1 85 | rv=$? 86 | else 87 | rv=$G_RETOK 88 | fi 89 | return $rv 90 | } 91 | 92 | # 93 | # @brief append given tool to tools array 94 | # @param tool name 95 | # @param requirement (optional): 1 - required (default), 0 - optional 96 | # 97 | tools_addTool_GV() { 98 | logging_debug $LINENO $"dodaje narzedzie: " "[$1]" 99 | 100 | # g_tools+=( "$1=${2:-1}" ) 101 | ___g_tools=( "${___g_tools[@]}" "$1=${2:-1}" ) 102 | } 103 | 104 | # 105 | # @brief perform tools presence verification 106 | # 107 | tools_verify_SO() { 108 | local ret=() 109 | local t='' 110 | 111 | for t in "$@"; do 112 | # obtain each tool's attributes 113 | local key=$(assoc_getKey_SO "$t") 114 | local mandatory=$(assoc_getValue_SO "$t") 115 | local group="$(assoc_getGroup_SO "$t")" 116 | 117 | local tool= 118 | local counter=0 119 | 120 | # iterate over group optionals 121 | for tool in ${key//|/ }; do 122 | local presence=1 123 | local entry= 124 | 125 | tools_verifyToolPresence "$tool" || presence=0 126 | entry="${tool}=${presence}" 127 | 128 | # if group definition is present prepend it to the entry 129 | [ -n "$group" ] && 130 | entry="${group}:${entry}" 131 | 132 | # append the entry to the array 133 | ret=( "${ret[@]}" "$entry" ) 134 | 135 | # increment detected counter if tool is present 136 | [ "$presence" -eq 1 ] && 137 | counter=$(( counter + 1 )) 138 | done 139 | 140 | # break if mandatory tool is missing 141 | [ "$mandatory" -eq 1 ] && [ "$counter" -eq 0 ] && 142 | return $G_RETFAIL 143 | done 144 | 145 | # shellcheck disable=SC2086 146 | echo ${ret[*]} 147 | } 148 | 149 | # 150 | # @brief check if given tool has been detected 151 | # @param tool name 152 | # 153 | tools_isDetected() { 154 | # this function can cope with that kind of input 155 | # shellcheck disable=SC2068 156 | local t= 157 | t="$(assoc_lookupValue_SO "$1" "${___g_tools[@]}" )" 158 | t=$(wrappers_ensureNumeric_SO "$t") 159 | [ "$t" -eq 1 ] 160 | } 161 | 162 | # 163 | # @brief get first available tool from given group 164 | # @param group name 165 | # @param 166 | # 167 | tools_getFirstAvailableFromGroup_SO() { 168 | local t='' 169 | for t in $(assoc_lookupGroupKeys_SO "${1:-none}" "${___g_tools[@]}"); do 170 | tools_isDetected "$t" && { 171 | echo "$t" 172 | return $G_RETOK 173 | } 174 | done 175 | 176 | # shellcheck disable=SC2086 177 | return $G_RETUNAV 178 | } 179 | 180 | # 181 | # @brief check if given tool belongs to given group 182 | # @param group name 183 | # @param tool name 184 | # 185 | tools_isInGroup() { 186 | local t='' 187 | for t in $(assoc_lookupGroupKeys_SO "${1:-none}" "${___g_tools[@]}"); do 188 | # shellcheck disable=SC2086 189 | [ "$t" = "${2:-none}" ] && return $G_RETOK 190 | done 191 | 192 | logging_info $LINENO "$2" $"nie znajduje sie w grupie" "$1" 193 | # shellcheck disable=SC2086 194 | return $G_RETUNAV 195 | } 196 | 197 | # 198 | # @brief return first detected tool from a given group 199 | # 200 | tools_isInGroupAndDetected() { 201 | local t='' 202 | for t in $(assoc_lookupGroupKeys_SO "${1:-none}" "${___g_tools[@]}"); do 203 | # shellcheck disable=SC2086 204 | [ "$t" = "${2:-none}" ] && 205 | tools_isDetected "$t" && 206 | return $G_RETOK 207 | done 208 | 209 | logging_info $LINENO \ 210 | "$2" $"nie znajduje sie w grupie" "$1" $", badz nie zostal wykryty" 211 | 212 | # shellcheck disable=SC2086 213 | return $G_RETUNAV 214 | } 215 | 216 | # 217 | # @brief returns the number of tools in the group 218 | # @param group name 219 | # 220 | tools_countGroupMembers_SO() { 221 | local a=( $(assoc_lookupGroupKeys_SO "${1:-none}" "${___g_tools[@]}") ) 222 | echo "${#a[*]}" 223 | } 224 | 225 | # 226 | # @brief returns the number of detected tools in the group 227 | # 228 | tools_countDetectedGroupMembers_SO() { 229 | local t='' 230 | local count=0 231 | for t in $(assoc_lookupGroupKeys_SO "${1:-none}" "${___g_tools[@]}"); do 232 | # shellcheck disable=SC2086 233 | tools_isDetected "$t" && count=$(( count + 1 )) 234 | done 235 | 236 | echo "$count" 237 | } 238 | 239 | # 240 | # @brief concat the tools array to single line 241 | # 242 | tools_toString_SO() { 243 | echo "${___g_tools[*]}" 244 | } 245 | 246 | # 247 | # @brief concat the tools array to list 248 | # 249 | tools_toList_SO() { 250 | ( IFS=$'\n'; echo "${___g_tools[*]}" ) 251 | } 252 | 253 | # 254 | # @brief concat the group members to single line 255 | # @param group name 256 | # 257 | tools_groupToString_SO() { 258 | assoc_lookupGroupKv_SO "$1" "${___g_tools[@]}" 259 | } 260 | 261 | # 262 | # @brief concat the group members to list 263 | # @param group name 264 | # 265 | tools_groupToList_SO() { 266 | local a=( $(assoc_lookupGroupKv_SO "$1" "${___g_tools[@]}") ) 267 | ( IFS=$'\n'; echo "${a[*]}" ) 268 | } 269 | 270 | # EOF 271 | -------------------------------------------------------------------------------- /tests/unit_tests/libnapi_logging_test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # force indendation settings 4 | # vim: ts=4 shiftwidth=4 expandtab 5 | 6 | ######################################################################## 7 | ######################################################################## 8 | ######################################################################## 9 | 10 | # Copyright (C) 2017 Tomasz Wisniewski aka 11 | # DAGON 12 | # 13 | # http://github.com/dagon666 14 | # http://pcarduino.blogspot.co.uk 15 | # 16 | # 17 | # This program is free software: you can redistribute it and/or modify 18 | # it under the terms of the GNU General Public License as published by 19 | # the Free Software Foundation, either version 3 of the License, or 20 | # (at your option) any later version. 21 | # 22 | # This program is distributed in the hope that it will be useful, 23 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 24 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 25 | # GNU General Public License for more details. 26 | # 27 | # You should have received a copy of the GNU General Public License 28 | # along with this program. If not, see . 29 | 30 | ######################################################################## 31 | ######################################################################## 32 | ######################################################################## 33 | 34 | # module dependencies 35 | . ../../libs/libnapi_retvals.sh 36 | . ../../libs/libnapi_wrappers.sh 37 | 38 | 39 | # fakes/mocks 40 | . mock/scpmocker.sh 41 | 42 | # module under test 43 | . ../../libs/libnapi_logging.sh 44 | 45 | setUp() { 46 | scpmocker_setUp 47 | 48 | # restore original values 49 | ___g_output=( 1 'none' 0 1 0 ) 50 | } 51 | 52 | tearDown() { 53 | scpmocker_tearDown 54 | } 55 | 56 | # 57 | # general function to test printing routines 58 | # 59 | _test_printers() { 60 | local printer="$1" 61 | local verbosity="$2" 62 | local str='empty line variable' 63 | local output= 64 | local lines=0 65 | 66 | output=$("logging_${printer}" '' "$str") 67 | assertTrue "$printer return value - always success" $? 68 | 69 | lines=$(echo "$output" | grep -c "0: $str") 70 | assertEquals "$printer with default verbosity" \ 71 | 0 "$lines" 72 | 73 | ___g_output[$___g_output_verbosity]="$verbosity" 74 | output=$("logging_${printer}" '' "$str") 75 | lines=$(echo "$output" | grep -c "0: $str") 76 | assertEquals "$printer with verbosity = $verbosity" \ 77 | 1 "$lines" 78 | 79 | output=$("logging_${printer}" 123 "$str") 80 | lines=$(echo "$output" | grep -c "123: $str") 81 | assertEquals "$printer with verbosity = $verbosity + line number" \ 82 | 1 "$lines" 83 | 84 | ___g_output[$___g_output_verbosity]=1 85 | } 86 | 87 | test_logging_debug() { 88 | _test_printers 'debug' 3 89 | } 90 | 91 | test_logging_info() { 92 | _test_printers 'info' 2 93 | } 94 | 95 | test_logging_warning() { 96 | local output= 97 | output=$(logging_warning "warning message" | grep -c "WARNING") 98 | assertEquals "warning message format" \ 99 | 1 "$output" 100 | 101 | ___g_output[$___g_output_verbosity]=0 102 | output=$(logging_warning "warning message" | grep -c "WARNING") 103 | assertEquals "warning message format" \ 104 | 0 "$output" 105 | } 106 | 107 | test_logging_error() { 108 | local output= 109 | output=$(logging_error "error message" 2>&1 | grep -c "ERROR") 110 | assertEquals "error message format 1" \ 111 | 1 "$output" 112 | 113 | ___g_output[$___g_output_verbosity]=0 114 | output=$(logging_error "error message" 2>&1 | grep -c "ERROR") 115 | assertEquals "error message format 2" \ 116 | 1 "$output" 117 | } 118 | 119 | test_logging_msg() { 120 | local output= 121 | output=$(logging_msg "message" | grep -c " - message") 122 | assertEquals "message format" \ 123 | 1 "$output" 124 | 125 | ___g_output[$___g_output_verbosity]=0 126 | output=$(logging_msg "message" | grep -c " - message") 127 | assertEquals "message format" \ 128 | 0 "$output" 129 | } 130 | 131 | test_logging_status() { 132 | local output= 133 | output=$(logging_status 'INFO' "message" | grep -c "INFO") 134 | assertEquals "status format" \ 135 | 1 "$output" 136 | 137 | ___g_output[$___g_output_verbosity]=0 138 | output=$(logging_status 'INFO' "message" | grep -c "INFO") 139 | assertEquals "message format" \ 140 | 0 $output 141 | } 142 | 143 | test_logging_blit_producesExpectedOutput() { 144 | ___g_output[$___g_output_forkid]=0 145 | ___g_output[$___g_output_msgcnt]=8 146 | local output= 147 | 148 | output=$(_logging_blit "some message") 149 | assertEquals "testing blit function and output format" \ 150 | "00:0008 some message" "$output" 151 | 152 | _logging_blit "abc" > /dev/null 153 | 154 | assertEquals "checking the fork id status" \ 155 | 0 "${___g_output[$___g_output_forkid]}" 156 | 157 | assertEquals "checking the msg cnt" \ 158 | 9 "${___g_output[$___g_output_msgcnt]}" 159 | } 160 | 161 | test_logging_setVerbosity_configuresGlobalValue() { 162 | local expected=0 163 | 164 | for i in {1..16}; do 165 | expected="$i" 166 | [ "$i" -gt 4 ] && expected=1 167 | 168 | logging_setVerbosity "$i" >/dev/null 169 | 170 | assertEquals "check level for $i requested" \ 171 | "$expected" "${___g_output[$___g_output_verbosity]}" 172 | done 173 | } 174 | 175 | test_logging_setMessageCounter_configuresGlobalValue() { 176 | for i in {1..16}; do 177 | logging_setMsgCounter "$i" 178 | assertEquals "check level for $i requested" \ 179 | "$i" "${___g_output[$___g_output_msgcnt]}" 180 | done 181 | } 182 | 183 | test_logging_setForkId_configuresGlobalValue() { 184 | for i in {1..16}; do 185 | logging_setForkId "$i" 186 | assertEquals "check level for $i requested" \ 187 | "$i" "${___g_output[$___g_output_forkid]}" 188 | done 189 | } 190 | 191 | test_logging_getVerbosity_returnsGlobalVariableValues() { 192 | for i in {1..16}; do 193 | ___g_output[$___g_output_verbosity]="$i" 194 | 195 | assertEquals "check value for $i" \ 196 | "$i" "$(logging_getVerbosity_SO)" 197 | done 198 | } 199 | 200 | test_logging_getMsgCounter_returnsGlobalVariableValues() { 201 | for i in {1..16}; do 202 | ___g_output[$___g_output_msgcnt]="$i" 203 | 204 | assertEquals "check value for $i" \ 205 | "$i" "$(logging_getMsgCounter_SO)" 206 | done 207 | } 208 | 209 | test_logging_getForkId_returnsGlobalVariableValues() { 210 | for i in {1..16}; do 211 | ___g_output[$___g_output_forkid]="$i" 212 | 213 | assertEquals "check value for $i" \ 214 | "$i" "$(logging_getForkId_SO)" 215 | done 216 | } 217 | 218 | test_logging_raiseLogOverwrite_setsGlobalFlag() { 219 | ___g_output[$___g_output_owrt]=0 220 | logging_raiseLogOverwrite 221 | assertEquals "check flag raised" \ 222 | 1 "${___g_output[$___g_output_owrt]}" 223 | 224 | logging_raiseLogOverwrite 225 | assertEquals "check flag raised (2nd attempt)" \ 226 | 1 "${___g_output[$___g_output_owrt]}" 227 | } 228 | 229 | test_logging_clearLogOverwrite_clearsGlobalFlag() { 230 | ___g_output[$___g_output_owrt]=1 231 | logging_clearLogOverwrite 232 | assertEquals "check flag cleared" \ 233 | 0 "${___g_output[$___g_output_owrt]}" 234 | 235 | logging_clearLogOverwrite 236 | assertEquals "check flag cleared (2nd attempt)" \ 237 | 0 "${___g_output[$___g_output_owrt]}" 238 | } 239 | 240 | test_logging_setLogFile_redirectsOutputToLogFile() { 241 | local logFile=$(mktemp -p "$SHUNIT_TMPDIR") 242 | local msg="message to be logged" 243 | 244 | ___g_output[$___g_output_owrt]=1 245 | logging_setLogFile "$logFile" 246 | 247 | logging_error "$msg" 248 | _logging_redirectToStdout 249 | 250 | assertEquals "check file contents" \ 251 | "00:0002 ERROR -> $msg" "$(<${logFile})" 252 | } 253 | 254 | # shunit call 255 | . shunit2 256 | 257 | -------------------------------------------------------------------------------- /tests/integration_tests/napi/xml_result.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import datetime 4 | import logging 5 | import re 6 | import xml.etree.ElementTree as ET 7 | 8 | 9 | class XmlResult(object): 10 | 11 | VERSION = '2.2.0.2399' 12 | NAPIID = 'NapiProjekt' 13 | URL = 'http://pobierz.napiprojekt.pl' 14 | 15 | def __init__(self, subtitles = None, cover = None, movieDetails = None): 16 | self.logger = logging.getLogger() 17 | self.subtitles = subtitles 18 | self.movieDetails = movieDetails 19 | 20 | # don't set the cover if no subs 21 | self.cover = cover if subtitles else None 22 | 23 | # set "success" flag if subs provided 24 | self.success = True if subtitles else False 25 | 26 | def _makeSubtitlesElement(self, parent): 27 | subtitles = ET.SubElement(parent, 'subtitles') 28 | subtitlesId = ET.SubElement(subtitles, 'id') 29 | subtitlesId.text = self.subtitles.getId() 30 | 31 | subtitlesHash = ET.SubElement(subtitles, 'subs_hash') 32 | subtitlesHash.text = self.subtitles.getHash() 33 | 34 | filesize = ET.SubElement(subtitles, 'filesize') 35 | filesize.text = str(self.subtitles.getSize()) 36 | 37 | author = ET.SubElement(subtitles, 'author') 38 | author.text = 'IntegrationTester' 39 | 40 | uploader = ET.SubElement(subtitles, 'uploader') 41 | uploader.text = 'IntegrationTester' 42 | 43 | uploadDate = ET.SubElement(subtitles, 'upload_date') 44 | uploadDate.text = datetime.datetime.now().strftime("%Y-%n-%d %H:%M:%S") 45 | 46 | self.logger.debug("Subs Id: [{}], Subs hash: [{}], Size: [{}]".format( 47 | subtitlesId.text, subtitlesHash.text, filesize.text)) 48 | 49 | contents = ET.SubElement(subtitles, 'content') 50 | contents.text = self._makeNapiCdata(self.subtitles) 51 | 52 | def _makeNapiCdata(self, blob): 53 | return self._makeNapiCdataString(blob.getBase64()) 54 | 55 | def _makeNapiCdataString(self, data): 56 | # !!! Hack Alert !!! 57 | # Original napi xml file holds CDATA in <> 58 | # which is probably wrong, but it's impossible to use these in 59 | # the element's test as they will be encoded into < and %gt; 60 | # 61 | # These custom markers will be replaced later on once the xml is 62 | # produced 63 | return '[OPEN_TAG]' + self._makeCdata(data) + '[CLOSE_TAG]' 64 | 65 | def _makeCdata(self, data): 66 | return '![CDATA[' + data + ']]' 67 | 68 | def _normalizeCdata(self, xmlStr): 69 | def tagReplace(mathObj): 70 | return ('<' if mathObj.group(0) == "[OPEN_TAG]" else '>') 71 | # - Excuse me Sir, is this the second part of the previously 72 | # mentioned hack? 73 | # - Good eye! Yes, it is, indeed! 74 | return re.sub(r'\[(OPEN|CLOSE)_TAG\]', 75 | tagReplace, xmlStr) 76 | 77 | def _makeResponseTime(self, parent): 78 | # fake response time 79 | responseTime = ET.SubElement(parent, 'response_time') 80 | responseTime.text = '0.08 s.' 81 | 82 | def _makeAdvertisment(self, parent): 83 | advertisment = ET.SubElement(parent, 'advertisment') 84 | adType = ET.SubElement(advertisment, 'type') 85 | adType.text = 'flash' 86 | 87 | location = ET.SubElement(advertisment, 'flash_location_url') 88 | location.text = 'http://www.napiprojekt.pl/banners/show.php?id=24' 89 | 90 | def _makeUpdateInfo(self, parent): 91 | updateInfo = ET.SubElement(parent, 'update_info') 92 | 93 | versionNumber = ET.SubElement(updateInfo, 'version_number') 94 | versionNumber.text = self.VERSION 95 | 96 | downloadUrl = ET.SubElement(updateInfo, 'download_url') 97 | downloadUrl.text = 'http://pobierz.napiprojekt.pl' 98 | 99 | latestChanges = ET.SubElement(updateInfo, 'latest_changes') 100 | latestChanges.text = """ 101 | NapiProjekt 2.2.0.2399 (2013-09-30) 102 | - Możliwość zdefiniowania kilku profilów pobieranych napisów 103 | - Program odporny na błędne nazwy folderów (zawierające np. niedopuszczone przez Windows znaki ':') 104 | - Instalator programu dodaje wyjątek do zapory Windows (naprawa komunikacji z serwerem w niektórych przypadkach) 105 | - Poprawione wyszukiwanie napisów na dysku 106 | 107 | 108 | NapiProjekt 2.1.1.2310 (2013-06-13) 109 | - Opcja automatycznego wyszukwiania napisów po uruchomieniu 'kolejki oczekujących' 110 | - Poprawiono błąd dotyczący znikania okienka 'kolejka oczekujących' 111 | - Wersja portable (plik ustawienia.ini nalezy przenieść do katalogu z programem) 112 | - Likiwidacja zgłoszonych błędów 113 | - Program nie sprawdza braku menu 'znajdź i dopasuj napisy' dla plików '*.oga, *.ogg, *.spx, *.ram, *.ogx, *.ra' 114 | - Usunięto zgłoszone błędy 115 | """ 116 | 117 | def _makeCover(self, parent): 118 | if self.cover: 119 | cover = ET.SubElement(parent, 'cover') 120 | cover.text = self._makeNapiCdata(self.cover) 121 | 122 | def _makeMovieDetails(self, parent): 123 | title = ET.SubElement(parent, 'title') 124 | title.text = self._makeNapiCdataString(self.movieDetails.title) 125 | 126 | otherTitle = ET.SubElement(parent, 'other_titles') 127 | otherTitle0 = ET.SubElement(otherTitle, 'other_0') 128 | otherTitle0.text = self._makeNapiCdataString(self.movieDetails.otherTitle) 129 | 130 | year = ET.SubElement(parent, 'year') 131 | year.text = self.movieDetails.year 132 | 133 | country = ET.SubElement(parent, 'country') 134 | countryPl = ET.SubElement(country, 'pl') 135 | countryPl.text = self.movieDetails.countryPl 136 | countryEn = ET.SubElement(country, 'en') 137 | countryEn.text = self.movieDetails.countryEn 138 | 139 | genre = ET.SubElement(parent, 'genre') 140 | genrePl = ET.SubElement(genre, 'pl') 141 | genrePl.text = self.movieDetails.genrePl 142 | genreEn = ET.SubElement(genre, 'en') 143 | genreEn.text = self.movieDetails.genreEn 144 | 145 | direction = ET.SubElement(parent, 'direction') 146 | direction.text = self.movieDetails.direction 147 | screenplay = ET.SubElement(parent, 'screenplay') 148 | screenplay.text = self.movieDetails.screenplay 149 | cinematography = ET.SubElement(parent, 'cinematography') 150 | cinematography.text = self.movieDetails.cinematography 151 | 152 | links = ET.SubElement(parent, 'direct_links') 153 | imdb = ET.SubElement(links, 'imdb_com') 154 | imdb.text = self._makeNapiCdataString(self.movieDetails.imdb) 155 | filmweb = ET.SubElement(links, 'filmweb_pl') 156 | filmweb.text = self._makeNapiCdataString(self.movieDetails.filmweb) 157 | fdb = ET.SubElement(links, 'fdb_pl') 158 | fdb.text = self._makeNapiCdataString(self.movieDetails.fdb) 159 | stopklatka = ET.SubElement(links, 'stopklatka_pl') 160 | stopklatka.text = self._makeNapiCdataString(self.movieDetails.stopklatka) 161 | onet = ET.SubElement(links, 'onet_pl') 162 | onet.text = self._makeNapiCdataString(self.movieDetails.onet) 163 | wp = ET.SubElement(links, 'wp_pl') 164 | wp.text = self._makeNapiCdataString(self.movieDetails.wp) 165 | 166 | rating = ET.SubElement(parent, 'rating') 167 | rating.text = self.movieDetails.rating 168 | votes = ET.SubElement(parent, 'votes') 169 | votes.text = self.movieDetails.votes 170 | 171 | 172 | def _makeMovie(self, parent): 173 | movie = ET.SubElement(parent, 'movie') 174 | self._makeStatus(movie) 175 | if self.success: 176 | self._makeCover(movie) 177 | 178 | if self.movieDetails: 179 | self._makeMovieDetails(movie) 180 | 181 | def _makeStatus(self, parent): 182 | status = ET.SubElement(parent, 'status') 183 | status.text = 'success' if self.success else 'failed' 184 | 185 | def toString(self): 186 | result = ET.Element('result') 187 | self._makeMovie(result) 188 | 189 | if self.success: 190 | self._makeStatus(result) 191 | self._makeSubtitlesElement(result) 192 | 193 | self._makeAdvertisment(result) 194 | self._makeUpdateInfo(result) 195 | self._makeResponseTime(result) 196 | return self._normalizeCdata(ET.tostring(result, 'utf-8')) 197 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # This project has been permanently moved to [gitlab](https://gitlab.com/hesperos/napi). It will no longer be maintained here. Please, refer to [gitlab](https://gitlab.com/hesperos/napi) repository for updates and new releases. 2 | 3 | [![pipeline status](https://gitlab.com/hesperos/napi/badges/master/pipeline.svg)](https://gitlab.com/hesperos/napi/commits/master) 4 | [![coverage report](https://gitlab.com/hesperos/napi/badges/master/coverage.svg)](https://gitlab.com/hesperos/napi/commits/master) 5 | 6 | # Bashnapi - napiprojekt.pl client 7 | 8 | This script is a [NapiProjekt](napiprojekt.pl) client written in bash. It 9 | automatically downloads subtitles from napiprojekt.pl database basing on the 10 | video file. 11 | 12 | This script works on Linux & OS X systems. It has very limited requirements and 13 | is mostly depending on bash (it is proven to run from bash 2.04 - which makes 14 | it ideal for embedded devices) and `coreutils` (which should be available on most 15 | modern systems, no Perl or Python is required). 16 | 17 | ## Installation 18 | 19 | **Bashnapi** uses [cmake](https://cmake.org) to build itself and install. 20 | Typical install procedure is very simple: 21 | 22 | $ cd bashnapi 23 | $ mkdir build && cd build 24 | $ cmake .. 25 | $ make && make install 26 | 27 | ### Installation on embedded devices 28 | 29 | In case you want to install **bashnapi** on a host which doesn't have 30 | CMake, the procedure is very similar. Just install to a local 31 | directory and deploy that to your device. Below is an example: 32 | 33 | $ cd bashnapi 34 | $ mkdir build 35 | $ cmake -DCMAKE_INSTALL_PREFIX=napi_install -DNAPI_INSTALL_PREFIX=/opt/napi .. 36 | $ make && make install 37 | 38 | **bashnapi** is now installed in the `napi_install` directory on your local 39 | machine. Just deploy that to your device (with `scp`, `ftp`, or whatever you 40 | prefer) and add the path to a directory under `/opt/napi/bin` to your 41 | `PATH`. The variables: 42 | 43 | - `CMAKE_INSTALL_PREFIX` - defines the directory on the host to which napi 44 | will be installed 45 | 46 | - `NAPI_INSTALL_PREFIX` - defines the directory on the target to which napi 47 | should be deployed 48 | 49 | You can use any directory names, `napi_install` and `/opt/napi` have been picked 50 | arbitrarily without any strict reason. 51 | 52 | ### Dockerized application 53 | 54 | `napi.sh` is available as well as a Dockerized application. In order to use it 55 | with docker, just build the container image: 56 | 57 | $ docker build -t napi . 58 | 59 | Once it's built it can be used through docker: 60 | 61 | $ docker run -v /media:/mnt -it napi scan /mnt 62 | 63 | The above command maps the directory `/media` to a directory `/mnt` in the 64 | container and invokes `napi.sh` scan action in container's `/mnt`. 65 | 66 | ## Actions 67 | 68 | Script functionality has been divided into actions. Each action implements a 69 | specific request type. Available actions: 70 | 71 | - scan - scan a directory (or a single file) and download subtitles for all 72 | found video files, 73 | - download - download subtitles using a "dc link" 74 | - search - search for a movie 75 | - subtitles - list subtitles for given movie 76 | 77 | Each action has its own command set and its own help system as well so, 78 | 79 | $ napi.sh scan --help 80 | 81 | ... and 82 | 83 | $ napi.sh download --help 84 | 85 | ... will produce different output. Try out help for different actions to learn 86 | about how to use them and what do they do. Generic options, shared by all 87 | actions are listed in the global help: 88 | 89 | $ napi.sh --help 90 | 91 | Below are some usage examples 92 | 93 | ### scan action 94 | 95 | This action is the equivalent of napi 1.X versions behaviour. It goes either 96 | through given directories or media files and, creates a media file list and 97 | tries to download subtitles for all found media files. 98 | 99 | Examples: 100 | 101 | - Download subtitles for `video_file.avi`: 102 | 103 | $ napi.sh scan video_file.avi 104 | 105 | - Iterate through all elements in current directory and try to download 106 | subtitles for them. If directory contains subdirectories - than the script will 107 | also iterate through all the files in subdirectories: 108 | 109 | $ napi.sh scan * 110 | 111 | - Try to find and download subtitles for all files in `movie_dir/` directory: 112 | 113 | $ napi.sh scan movie_dir/ 114 | 115 | - This will recursively search for video file in directories like: 116 | 117 | $ napi.sh scan dir1/ dir2/ dir3/ dir_other/ 118 | 119 | - It has file size limitation too ! Download subtitles for all supported video 120 | files which are bigger than 100 MB: 121 | 122 | $ napi.sh scan -b 100 * 123 | 124 | - Not to mention that it integrates a separate subtitles converter written 125 | completely in **bash** & **awk**. To download subtitles for all supported video 126 | files and convert them to subrip format on the fly, just use the **-f** option: 127 | 128 | $ napi.sh -f subrip * 129 | 130 | ### download action (experimental) 131 | 132 | This action can be used to download a selected subtitles from napiprojekt.pl 133 | using the subtitles id, which can be obtained from napiprojekt.pl site. 134 | 135 | - Download subtitles having its hash/id: 136 | 137 | $ napi.sh download napiprojekt:06aec10a749a68403613b2af8b2c4db8 138 | 139 | ### search action (experimental) 140 | 141 | This action can be used to search for a given movie in napiprojekt.pl database. 142 | 143 | - Search for movie "terminator": 144 | 145 | 146 | $ napi.sh search -k movie terminator 147 | $ napi.sh search "the big bang theory" 148 | 149 | ### subtitles action (experimental) 150 | 151 | This action can be used to list all the available subtitles for a given movie 152 | title. It accepts the url to the movie page, typical work flow is as follows: 153 | 154 | 155 | $ napi.sh search -k movie "lord of the rings" 156 | 157 | ``` 158 | ... 159 | 00:0003 - Wyszukuje tytul: [hobbit] 160 | 29516 | Hobbit: Niezwykła podróż | http://napiprojekt.pl/napisy-29516-Hobbit-Niezwykła-podróż-(2012) 161 | 37789 | Hobbit: Pustkowie Smauga | http://napiprojekt.pl/napisy-37789-Hobbit-Pustkowie-Smauga-(2013) 162 | 44148 | Hobbit: Bitwa Pięciu Armii | http://napiprojekt.pl/napisy-44148-Hobbit-Bitwa-Pięciu-Armii-(2014) 163 | 162 | Hobbit | http://napiprojekt.pl/napisy-162-Hobbit-(1977) 164 | ``` 165 | 166 | $ napi.sh subtitles "http://napiprojekt.pl/napisy-29516-Hobbit-Niezwykła-podróż-(2012)" 167 | 168 | ``` 169 | ... 170 | 00:0003 - Przetwarzam: [http://napiprojekt.pl/napisy-29516-Hobbit-Niezwykła-podróż-(2012)] 171 | Rozmiar: 2491657374 bajtow | fps: 29.534 | napiprojekt:f2bed6d99e5ecc9d7b2b3cb7c51c273e 172 | Rozmiar: 1608081408 bajtow | fps: 29.534 | napiprojekt:1e81de9b83485336d2821d8dcfefb8bd 173 | Rozmiar: 1742344634 bajtow | fps: 29.534 | napiprojekt:51f8741fc142f3ed80313544c728d9d4 174 | Rozmiar: 1442403786 bajtow | fps: 29.534 | napiprojekt:ee4096dce1902ea5f985dc929c9a8479 175 | Rozmiar: 782801208 bajtow | fps: 29.535 | napiprojekt:7fd27d9777eea21f7a2b92c10919c43c 176 | ... 177 | ``` 178 | 179 | The last call has returned a set of napiprojekt subtitles identifiers which can 180 | be directly used to get subtitles: 181 | 182 | $ napi.sh download napiprojekt:1e81de9b83485336d2821d8dcfefb8bd 183 | 184 | You can specify more than one hash/id at once as well: 185 | 186 | $ napi.sh download napiprojekt:51f8741fc142f3ed80313544c728d9d4 napiprojekt:ee4096dce1902ea5f985dc929c9a8479 187 | 188 | 189 | ## subotage.sh 190 | 191 | `subotage.sh` is a simple subtitles format converter bundled with `napi.sh` 192 | 193 | Currently supported formats: 194 | - mpl2 195 | - tmplayer (most of the versions) 196 | - subrip 197 | - subviewer 198 | - microdvd 199 | 200 | ### Usage 201 | 202 | The properly convert from/to microdvd format (or any other format based on 203 | frames) a valid information about input/output file frame rate is 204 | needed! The default value (if not specified in the command line) is 23.98 fps 205 | for input/output. 206 | 207 | Examples: 208 | 209 | - Convert from microdvd 23.98 fps to subrip. Subrip is default output format so 210 | it doesn't have to be specified. The input frame rate is also equal to the 211 | default one, so no addition specification in the command line has been made. 212 | 213 | 214 | $ subotage.sh -i input_file.txt -o output_file.srt 215 | 216 | 217 | - Convert from microdvd 25 fps to subviewer: 218 | 219 | 220 | $ subotage.sh -i input_file.txt -fi 25 -of subviewer -o output_file.sub 221 | 222 | 223 | - Convert from subrip to mpl2 224 | 225 | 226 | $ subotage.sh -i input_file.srt -of mpl2 -o output_file.fab 227 | 228 | 229 | - Convert from microdvd 25 fps to microdvd 29.98 fps: 230 | 231 | 232 | $ subotage.sh -i input_file.txt -fi 25 -fo 29.98 -of microdvd -o output_file.txt 233 | 234 | 235 | # Colaboration 236 | 237 | **bashnapi** is an open project. Feel free to send patches and pull requests. 238 | Check the [COLABORATION](COLABORATION.md) for more details. 239 | --------------------------------------------------------------------------------