├── .dockerignore ├── .gitignore ├── Dockerfile ├── Dockerfile.armv7l ├── Dockerfile.nginx ├── Dockerfile.worker ├── Images ├── default.png ├── reader.png ├── settings.png └── show_bookmarks.png ├── LICENSE ├── README.md ├── accounts ├── __init__.py ├── admin.py ├── apps.py ├── forms.py ├── models.py ├── templates │ ├── nosignup.html │ └── signup.html └── views.py ├── docker-compose.yml ├── docker.env ├── hlspy.env ├── manage.py ├── nginx.conf ├── pages ├── __init__.py ├── admin.py ├── apps.py ├── custom_read.py ├── dbaccess.py ├── forms.py ├── management │ └── commands │ │ ├── __init__.py │ │ ├── applysettings.py │ │ ├── createdefaultsu.py │ │ ├── generatesecretkey.py │ │ └── nltkdownload.py ├── migrations │ ├── 0001_initial.py │ ├── 0002_auto_20180718_1649.py │ ├── 0003_auto_20180718_1650.py │ ├── 0004_auto_20180722_1016.py │ ├── 0005_usersettings_total_tags.py │ ├── 0006_usersettings_buddy_list.py │ ├── 0007_auto_20180723_0944.py │ ├── 0008_auto_20180726_1216.py │ ├── 0009_usersettings_png_quality.py │ ├── 0010_usersettings_auto_archieve.py │ ├── 0011_library_tags.py │ ├── 0012_library_icon_url.py │ ├── 0013_auto_20180805_1521.py │ ├── 0014_auto_20180809_1307.py │ ├── 0015_usersettings_pagination_value.py │ ├── 0016_auto_20180925_1749.py │ ├── 0017_library_reader_mode.py │ ├── 0018_auto_20181011_0532.py │ ├── 0019_usersettings_reader_theme.py │ ├── 0020_library_subdir.py │ ├── 0021_auto_20181208_1553.py │ ├── 0022_auto_20181208_2051.py │ └── __init__.py ├── models.py ├── summarize.py ├── urls.py ├── utils.py └── views.py ├── reminiscence ├── __init__.py ├── celery.py ├── defaultsettings.py ├── dockersettings.py ├── settings.py ├── urls.py └── wsgi.py ├── requirements.txt ├── restapi ├── __init__.py ├── admin.py ├── apps.py ├── migrations │ └── __init__.py ├── models.py ├── tests.py ├── urls.py └── views.py ├── static ├── archive.svg ├── css │ ├── accounts.css │ ├── bootstrap.min.css │ ├── bootstrap.min.css.map │ ├── summernote-bs4.css │ └── text_layer_builder.css ├── external-link.svg ├── folder.svg ├── img │ └── full-bloom.png ├── js │ ├── annotator.min.js │ ├── bootbox.min.js │ ├── bootstrap.min.js │ ├── epub.min.js │ ├── jquery-3.3.1.min.js │ ├── main.js │ ├── pdf.min.js │ ├── pdf.worker.min.js │ ├── popper.min.js │ └── summernote-bs4.js └── menu.svg ├── templates ├── archive_not_found.html ├── base.html ├── home.html ├── home_dir.html ├── includes │ └── form.html ├── login.html ├── password_change.html ├── password_change_done.html └── public.html ├── tests ├── __init__.py ├── tests_drf.py ├── tests_home.py ├── tests_signup.py └── tests_sync.py └── vinanti ├── __init__.py ├── crawl.py ├── formdata.py ├── log.py ├── req.py ├── req_aio.py ├── req_urllib.py ├── utils.py └── vinanti.py /.dockerignore: -------------------------------------------------------------------------------- 1 | **/.git 2 | **/archive 3 | **/static 4 | **/logs 5 | **/db 6 | **/tmp 7 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.log 2 | *.sqlite3 3 | *.pyc 4 | __pycache__/ 5 | archive/ 6 | static/favicons/ 7 | static/nltk_data/ 8 | db/ 9 | logs/ 10 | tmp/ 11 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.10-slim-bookworm 2 | 3 | WORKDIR /usr/src/reminiscence 4 | 5 | RUN apt-get update \ 6 | && apt-get install --no-install-recommends -y netcat-traditional htop \ 7 | && rm -rf /var/lib/apt/lists/* 8 | 9 | COPY requirements.txt . 10 | 11 | RUN pip install -r requirements.txt 12 | 13 | COPY . /usr/src/reminiscence 14 | 15 | RUN mkdir -p logs archive tmp \ 16 | && python manage.py applysettings --docker yes \ 17 | && python manage.py generatesecretkey 18 | -------------------------------------------------------------------------------- /Dockerfile.armv7l: -------------------------------------------------------------------------------- 1 | FROM python:3.11-slim-bookworm 2 | 3 | WORKDIR /usr/src/reminiscence 4 | 5 | RUN apt-get update \ 6 | && apt-get install --no-install-recommends -y \ 7 | build-essential \ 8 | libpq-dev \ 9 | libxml2 \ 10 | libxml2-dev \ 11 | libxslt1-dev \ 12 | python-dev-is-python3 \ 13 | python3-pyqt5 \ 14 | python3-pyqt5.qtwebengine \ 15 | libpython3-all-dev \ 16 | zlib1g-dev \ 17 | chromium \ 18 | netcat-traditional \ 19 | git \ 20 | htop \ 21 | && rm -rf /var/lib/apt/lists/* 22 | 23 | COPY requirements.txt . 24 | 25 | RUN pip install -r requirements.txt 26 | 27 | RUN pip install git+https://github.com/kanishka-linux/hlspy 28 | 29 | COPY . /usr/src/reminiscence 30 | 31 | RUN bash 32 | 33 | RUN mkdir -p logs archive tmp \ 34 | && python manage.py applysettings --docker yes \ 35 | && python manage.py generatesecretkey 36 | -------------------------------------------------------------------------------- /Dockerfile.nginx: -------------------------------------------------------------------------------- 1 | FROM nginx:latest 2 | COPY ./nginx.conf /etc/nginx/nginx.conf 3 | -------------------------------------------------------------------------------- /Dockerfile.worker: -------------------------------------------------------------------------------- 1 | FROM python:3.10-slim-bookworm 2 | 3 | WORKDIR /usr/src/reminiscence 4 | 5 | RUN apt-get update \ 6 | && apt-get install --no-install-recommends -y chromium netcat-traditional git htop \ 7 | && rm -rf /var/lib/apt/lists/* 8 | 9 | COPY requirements.txt . 10 | 11 | RUN pip install -r requirements.txt 12 | 13 | RUN pip install PyQt5 PyQtWebEngine sip git+https://github.com/kanishka-linux/hlspy 14 | 15 | COPY . /usr/src/reminiscence 16 | 17 | RUN mkdir -p logs archive tmp \ 18 | && python manage.py applysettings --docker yes \ 19 | -------------------------------------------------------------------------------- /Images/default.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/Images/default.png -------------------------------------------------------------------------------- /Images/reader.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/Images/reader.png -------------------------------------------------------------------------------- /Images/settings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/Images/settings.png -------------------------------------------------------------------------------- /Images/show_bookmarks.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/Images/show_bookmarks.png -------------------------------------------------------------------------------- /accounts/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/accounts/__init__.py -------------------------------------------------------------------------------- /accounts/admin.py: -------------------------------------------------------------------------------- 1 | from django.contrib import admin 2 | 3 | # Register your models here. 4 | -------------------------------------------------------------------------------- /accounts/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | 3 | 4 | class AccountsConfig(AppConfig): 5 | name = 'accounts' 6 | -------------------------------------------------------------------------------- /accounts/forms.py: -------------------------------------------------------------------------------- 1 | from django import forms 2 | from django.contrib.auth.forms import UserCreationForm 3 | from django.contrib.auth.models import User 4 | 5 | class SignUpForm(UserCreationForm): 6 | email = forms.CharField(max_length=254, required=True, widget=forms.EmailInput()) 7 | class Meta: 8 | model = User 9 | fields = ('username', 'email', 'password1', 'password2') 10 | -------------------------------------------------------------------------------- /accounts/models.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | 3 | # Create your models here. 4 | -------------------------------------------------------------------------------- /accounts/templates/nosignup.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% load static %} 4 | 5 | {% block stylesheet %} 6 | 7 | {% endblock %} 8 | 9 | {% block body %} 10 |
11 |

12 | Reminiscence 13 |

14 |
15 |
16 | 19 | 22 |
23 |
24 |
25 | {% endblock %} 26 | -------------------------------------------------------------------------------- /accounts/templates/signup.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% load static %} 4 | 5 | {% block stylesheet %} 6 | 7 | {% endblock %} 8 | 9 | {% block body %} 10 |
11 |

12 | Reminiscence 13 |

14 |
15 |
16 |
17 |
18 |
19 | {% csrf_token %} 20 | {% include 'includes/form.html' %} 21 | 22 |
23 |
24 |
25 | Already have an account? Log in 26 |
27 |
28 |
29 |
30 |
31 | {% endblock %} 32 | -------------------------------------------------------------------------------- /accounts/views.py: -------------------------------------------------------------------------------- 1 | from django.contrib.auth import login as auth_login 2 | from django.contrib.auth.forms import UserCreationForm 3 | from django.shortcuts import render, redirect 4 | from django.conf import settings 5 | from django.http import HttpResponse 6 | 7 | def signup(request): 8 | if settings.ALLOW_ANY_ONE_SIGNUP: 9 | if request.method == 'POST': 10 | form = UserCreationForm(request.POST) 11 | if form.is_valid(): 12 | user = form.save() 13 | auth_login(request, user) 14 | return redirect('home') 15 | else: 16 | form = UserCreationForm() 17 | return render(request, 'signup.html', {'form': form}) 18 | else: 19 | return render(request, 'nosignup.html') 20 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3' 2 | 3 | services: 4 | nginx: 5 | build: 6 | context: . 7 | dockerfile: Dockerfile.nginx 8 | volumes: 9 | - .:/usr/src/reminiscence 10 | ports: 11 | - "80:80" 12 | depends_on: 13 | - web 14 | 15 | web: 16 | build: . 17 | command: bash -c "while ! nc -w 1 -z db 5432; do sleep 0.1; done; python manage.py migrate; python manage.py createdefaultsu; python manage.py collectstatic --no-input; if [ ! -d '/usr/src/reminiscence/static/nltk_data' ]; then echo 'wait..downloading..nltk_data'; python manage.py nltkdownload; fi; gunicorn --max-requests 1000 --worker-class gthread --workers 4 --thread 10 --timeout 300 --bind 0.0.0.0:8000 reminiscence.wsgi" 18 | env_file: 19 | - ./docker.env 20 | ports: 21 | - "8000:8000" 22 | volumes: 23 | - ./static:/usr/src/reminiscence/static/ 24 | - ./archive:/usr/src/reminiscence/archive/ 25 | - ./logs:/usr/src/reminiscence/logs/ 26 | depends_on: 27 | - db 28 | 29 | worker: 30 | build: 31 | context: . 32 | dockerfile: Dockerfile.worker 33 | depends_on: 34 | - db 35 | - redis 36 | - web 37 | env_file: 38 | - ./docker.env 39 | volumes: 40 | - ./static:/usr/src/reminiscence/static/ 41 | - ./archive:/usr/src/reminiscence/archive/ 42 | - ./logs:/usr/src/reminiscence/logs/ 43 | command: bash -c "celery -A reminiscence worker --loglevel=info -c 4" 44 | 45 | db: 46 | image: postgres:13 47 | env_file: 48 | - ./docker.env 49 | # instead of using the env_file above for providing db-user-credentials 50 | # you also could use the following insecure setting (not recommended) 51 | # may come handy if you're experiencing db-connection problems after upgrade. 52 | #environment: 53 | # - POSTGRES_HOST_AUTH_METHOD=trust 54 | volumes: 55 | - ./db:/var/lib/postgresql/data/ 56 | 57 | redis: 58 | image: redis:7.2 59 | ports: 60 | - '6379:6379' 61 | -------------------------------------------------------------------------------- /docker.env: -------------------------------------------------------------------------------- 1 | POSTGRES_DB=postgres 2 | POSTGRES_USER=postgres 3 | POSTGRES_PASSWORD=password 4 | 5 | BROKER_URL='redis://redis:6379/0' 6 | CELERY_BROKER_URL='redis://redis:6379/0' 7 | CELERY_RESULT_BACKEND='redis://redis:6379/0' 8 | CELERY_BACKEND='redis://redis:6379/0' 9 | 10 | QT_QPA_PLATFORM=offscreen 11 | QTWEBENGINE_DISABLE_SANDBOX=1 12 | QT_QUICK_BACKEND=software 13 | QT_OPENGL=software 14 | 15 | PYTHONPATH=/usr/lib/python3/dist-packages 16 | -------------------------------------------------------------------------------- /hlspy.env: -------------------------------------------------------------------------------- 1 | export QT_QPA_PLATFORM=offscreen 2 | export QT_QUICK_BACKEND=software 3 | export QT_OPENGL=software 4 | -------------------------------------------------------------------------------- /manage.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import os 3 | import sys 4 | 5 | if __name__ == "__main__": 6 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "reminiscence.settings") 7 | try: 8 | from django.core.management import execute_from_command_line 9 | except ImportError as exc: 10 | raise ImportError( 11 | "Couldn't import Django. Are you sure it's installed and " 12 | "available on your PYTHONPATH environment variable? Did you " 13 | "forget to activate a virtual environment?" 14 | ) from exc 15 | execute_from_command_line(sys.argv) 16 | -------------------------------------------------------------------------------- /nginx.conf: -------------------------------------------------------------------------------- 1 | worker_processes 16; 2 | 3 | events { 4 | worker_connections 1024; 5 | } 6 | 7 | 8 | http { 9 | include mime.types; 10 | default_type application/octet-stream; 11 | 12 | sendfile on; 13 | sendfile_max_chunk 512k; 14 | keepalive_timeout 65; 15 | proxy_read_timeout 300s; 16 | upstream web_server { 17 | server web:8000; 18 | } 19 | server { 20 | listen 80; 21 | server_name localhost; 22 | client_max_body_size 1024m; 23 | 24 | location /static/ { 25 | root /usr/src/reminiscence; 26 | aio threads; 27 | } 28 | location = /favicon.ico { access_log off; log_not_found off; } 29 | location / { 30 | proxy_pass http://web_server; 31 | proxy_set_header Host $host; 32 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 33 | } 34 | 35 | 36 | error_page 500 502 503 504 /50x.html; 37 | location = /50x.html { 38 | root /usr/share/nginx/html; 39 | } 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /pages/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/pages/__init__.py -------------------------------------------------------------------------------- /pages/admin.py: -------------------------------------------------------------------------------- 1 | from django.contrib import admin 2 | 3 | # Register your models here. 4 | -------------------------------------------------------------------------------- /pages/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | 3 | 4 | class PagesConfig(AppConfig): 5 | name = 'pages' 6 | -------------------------------------------------------------------------------- /pages/forms.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of Reminiscence. 5 | 6 | Reminiscence is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Affero General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | Reminiscence is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Affero General Public License for more details. 15 | 16 | You should have received a copy of the GNU Affero General Public License 17 | along with Reminiscence. If not, see . 18 | """ 19 | 20 | import re 21 | import logging 22 | from django import forms 23 | from django.utils import timezone 24 | from .models import Library 25 | from .dbaccess import DBAccess as dbxs 26 | 27 | logger = logging.getLogger(__name__) 28 | 29 | 30 | class AddDir(forms.Form): 31 | 32 | create_directory = forms.CharField( 33 | max_length=2048, required=True, 34 | widget=forms.TextInput(attrs={'placeholder':'Create New Directory'}) 35 | ) 36 | DEFAULT_DIRECTORY = 'Bookmarks' 37 | 38 | def clean_and_save_data(self, usr): 39 | dirname = self.cleaned_data.get('create_directory') 40 | http = re.match(r'^(?:http)s?://(?!/)', dirname) 41 | if http: 42 | url = dirname 43 | qdir = Library.objects.filter(usr=usr, 44 | directory=self.DEFAULT_DIRECTORY) 45 | logger.info('adding {} to Bookmark'.format(url)) 46 | if not qdir and len(url) > 9: 47 | Library.objects.create(usr=usr, directory=self.DEFAULT_DIRECTORY, timestamp=timezone.now()).save() 48 | dbxs.process_add_url.delay( 49 | usr.id, url, 50 | self.DEFAULT_DIRECTORY,False 51 | ) 52 | logger.debug('add--bookmark') 53 | elif qdir and len(url) > 9: 54 | nqdir = qdir.filter(url=url) 55 | if not nqdir: 56 | dbxs.process_add_url.delay( 57 | usr, url, 58 | self.DEFAULT_DIRECTORY, False 59 | ) 60 | else: 61 | dirname = re.sub(r'/|:|#|\?|\\\\|\%', '-', dirname) 62 | if dirname: 63 | qdir = Library.objects.filter(usr=usr, directory=dirname) 64 | if not qdir: 65 | Library.objects.create(usr=usr, directory=dirname, timestamp=timezone.now()).save() 66 | 67 | 68 | class AddURL(forms.Form): 69 | add_url = forms.URLField( 70 | max_length=2048, required=True, 71 | widget=forms.TextInput(attrs={'placeholder':'Enter URL'}) 72 | ) 73 | 74 | 75 | class RenameDir(forms.Form): 76 | rename_directory = forms.CharField( 77 | max_length=200, required=True, 78 | widget=forms.TextInput(attrs={'placeholder':'Enter New Name'}) 79 | ) 80 | 81 | def clean_and_rename(self, usr, directory): 82 | ren_dir = self.cleaned_data.get('rename_directory') 83 | if ren_dir and ren_dir != directory: 84 | ren_dir = re.sub(r'/|:|#|\?|\\\\|\%', '-', ren_dir) 85 | if '/' in directory: 86 | dbxs.remove_subdirectory_link(usr, directory, ren_dir) 87 | pdir, _ = directory.rsplit('/', 1) 88 | ren_dir = pdir + '/' + ren_dir 89 | Library.objects.filter(usr=usr, directory=directory).update(directory=ren_dir) 90 | qlist = Library.objects.filter(usr=usr, directory__istartswith=directory+'/') 91 | for row in qlist: 92 | row.directory = re.sub(directory, ren_dir, row.directory, 1) 93 | row.save() 94 | 95 | 96 | class RemoveDir(forms.Form): 97 | CHOICES = ( 98 | (False, 'Do Not Remove'), 99 | (True, 'Remove') 100 | ) 101 | remove_directory = forms.BooleanField(widget=forms.Select(choices=CHOICES)) 102 | 103 | def check_and_remove_dir(self, usr, directory): 104 | rem_dir = self.cleaned_data.get('remove_directory', '') 105 | if rem_dir is True: 106 | qlist = Library.objects.filter(usr=usr, directory=directory) 107 | for row in qlist: 108 | dbxs.remove_url_link(usr, row=row) 109 | qlist = Library.objects.filter(usr=usr, directory__istartswith=directory+'/') 110 | for row in qlist: 111 | dbxs.remove_url_link(usr, row=row) 112 | if '/' in directory: 113 | dbxs.remove_subdirectory_link(usr, directory) 114 | 115 | -------------------------------------------------------------------------------- /pages/management/commands/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/pages/management/commands/__init__.py -------------------------------------------------------------------------------- /pages/management/commands/applysettings.py: -------------------------------------------------------------------------------- 1 | import os 2 | import reminiscence 3 | from django.conf import settings 4 | from django.core import management 5 | import shutil 6 | 7 | BASE_DIR = os.path.dirname(reminiscence.__file__) 8 | 9 | class Command(management.BaseCommand): 10 | 11 | help = 'Apply docker or default settings' 12 | 13 | def add_arguments(self, parser): 14 | super(Command, self).add_arguments(parser) 15 | parser.add_argument( 16 | '--docker', dest='docker', default=None, 17 | help='Apply docker specific settings', 18 | ) 19 | parser.add_argument( 20 | '--default', dest='default', default=None, 21 | help='Apply default settings', 22 | ) 23 | 24 | def handle(self, *args, **options): 25 | original = os.path.join(BASE_DIR, 'settings.py') 26 | dock = os.path.join(BASE_DIR, 'dockersettings.py') 27 | default = os.path.join(BASE_DIR, 'defaultsettings.py') 28 | optdock = options.get('docker') 29 | optdef = options.get('default') 30 | if optdock and optdock.lower() == 'yes': 31 | shutil.copy(dock, original) 32 | print('docker settings copied') 33 | elif optdef and optdef.lower() == 'yes': 34 | shutil.copy(default, original) 35 | print('default settings copied') 36 | -------------------------------------------------------------------------------- /pages/management/commands/createdefaultsu.py: -------------------------------------------------------------------------------- 1 | from django.core import management 2 | from django.contrib.auth.models import User 3 | 4 | 5 | class Command(management.BaseCommand): 6 | 7 | def handle(self, *args, **options): 8 | qlist = User.objects.filter(username='admin') 9 | if not qlist: 10 | print('creating default superuser: "admin" with password: "changepassword"') 11 | User.objects.create_superuser('admin', 'admin@reminiscence.org', 'changepassword') 12 | else: 13 | print('default admin already exists') 14 | -------------------------------------------------------------------------------- /pages/management/commands/generatesecretkey.py: -------------------------------------------------------------------------------- 1 | import os 2 | import re 3 | import reminiscence 4 | from django.conf import settings 5 | from django.core import management 6 | from django.utils.crypto import get_random_string 7 | from tempfile import mkstemp 8 | import shutil 9 | 10 | BASE_DIR = os.path.dirname(reminiscence.__file__) 11 | 12 | class Command(management.BaseCommand): 13 | help = 'Generates a random secret key.' 14 | 15 | @staticmethod 16 | def _generate_secret_key(): 17 | chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()-_=+' 18 | return get_random_string(50, chars) 19 | 20 | def handle(self, *args, **options): 21 | orig = os.path.join(BASE_DIR, 'settings.py') 22 | fd, temp = mkstemp() 23 | shutil.copy(orig, temp) 24 | 25 | with open(temp, 'w') as new_file: 26 | with open(orig) as old_file: 27 | for line in old_file: 28 | secret_key = re.match(r'^SECRET_KEY ?=', line) 29 | if secret_key: 30 | line = "SECRET_KEY = '{0}'".format(Command._generate_secret_key()) + '\n' 31 | new_file.write(line) 32 | 33 | new_file.close() 34 | shutil.copy(temp, orig) 35 | os.close(fd) 36 | os.remove(temp) 37 | -------------------------------------------------------------------------------- /pages/management/commands/nltkdownload.py: -------------------------------------------------------------------------------- 1 | import os 2 | from django.conf import settings 3 | from django.core import management 4 | import nltk 5 | 6 | 7 | class Command(management.BaseCommand): 8 | 9 | nltk_data_path = settings.NLTK_DATA_PATH 10 | nltk.data.path.append(nltk_data_path) 11 | 12 | def handle(self, *args, **options): 13 | if not os.path.exists(self.nltk_data_path): 14 | os.makedirs(self.nltk_data_path) 15 | nltk.download( 16 | [ 17 | 'stopwords', 'punkt', 18 | 'averaged_perceptron_tagger' 19 | ], download_dir=self.nltk_data_path 20 | ) 21 | -------------------------------------------------------------------------------- /pages/migrations/0001_initial.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-18 16:49 2 | 3 | from django.conf import settings 4 | from django.db import migrations, models 5 | import django.db.models.deletion 6 | 7 | 8 | class Migration(migrations.Migration): 9 | 10 | initial = True 11 | 12 | dependencies = [ 13 | migrations.swappable_dependency(settings.AUTH_USER_MODEL), 14 | ] 15 | 16 | operations = [ 17 | migrations.CreateModel( 18 | name='Library', 19 | fields=[ 20 | ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), 21 | ('directory', models.CharField(max_length=2048)), 22 | ('url', models.CharField(max_length=4096, null=True)), 23 | ('title', models.CharField(max_length=2048, null=True)), 24 | ('timestamp', models.DateTimeField(auto_now_add=True, null=True)), 25 | ('media_path', models.CharField(max_length=4096, null=True)), 26 | ('usr', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='usr', to=settings.AUTH_USER_MODEL)), 27 | ], 28 | ), 29 | migrations.CreateModel( 30 | name='Tags', 31 | fields=[ 32 | ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), 33 | ('tag', models.CharField(max_length=100)), 34 | ], 35 | ), 36 | migrations.CreateModel( 37 | name='URLTags', 38 | fields=[ 39 | ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), 40 | ('tag_id', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='tag_name', to='pages.Tags')), 41 | ('url_id', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='url_library', to='pages.Library')), 42 | ('usr_id', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='usr_tag', to=settings.AUTH_USER_MODEL)), 43 | ], 44 | ), 45 | ] 46 | -------------------------------------------------------------------------------- /pages/migrations/0002_auto_20180718_1649.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-18 16:49 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0001_initial'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='library', 15 | name='access', 16 | field=models.PositiveSmallIntegerField(choices=[(0, 'Public'), (1, 'Private'), (2, 'Group')], default=1), 17 | ), 18 | migrations.AddField( 19 | model_name='library', 20 | name='summary', 21 | field=models.TextField(null=True), 22 | ), 23 | ] 24 | -------------------------------------------------------------------------------- /pages/migrations/0003_auto_20180718_1650.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-18 16:50 2 | 3 | from django.conf import settings 4 | from django.db import migrations, models 5 | import django.db.models.deletion 6 | 7 | 8 | class Migration(migrations.Migration): 9 | 10 | dependencies = [ 11 | migrations.swappable_dependency(settings.AUTH_USER_MODEL), 12 | ('pages', '0002_auto_20180718_1649'), 13 | ] 14 | 15 | operations = [ 16 | migrations.CreateModel( 17 | name='GroupTable', 18 | fields=[ 19 | ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), 20 | ('buddy', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='usr_buddy', to=settings.AUTH_USER_MODEL)), 21 | ], 22 | ), 23 | migrations.CreateModel( 24 | name='UserSettings', 25 | fields=[ 26 | ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), 27 | ('autotag', models.BooleanField(default=False)), 28 | ('auto_summary', models.BooleanField(default=False)), 29 | ('usrid', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='usr_settings', to=settings.AUTH_USER_MODEL)), 30 | ], 31 | ), 32 | migrations.AddField( 33 | model_name='grouptable', 34 | name='user_set', 35 | field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='usr_set', to='pages.UserSettings'), 36 | ), 37 | ] 38 | -------------------------------------------------------------------------------- /pages/migrations/0004_auto_20180722_1016.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-22 10:16 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0003_auto_20180718_1650'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AlterField( 14 | model_name='tags', 15 | name='tag', 16 | field=models.CharField(max_length=100, unique=True), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0005_usersettings_total_tags.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-22 14:28 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0004_auto_20180722_1016'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='usersettings', 15 | name='total_tags', 16 | field=models.PositiveSmallIntegerField(default=5), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0006_usersettings_buddy_list.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-22 17:11 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0005_usersettings_total_tags'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='usersettings', 15 | name='buddy_list', 16 | field=models.CharField(max_length=8192, null=True), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0007_auto_20180723_0944.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-23 09:44 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0006_usersettings_buddy_list'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='usersettings', 15 | name='group_dir', 16 | field=models.CharField(max_length=2048, null=True), 17 | ), 18 | migrations.AddField( 19 | model_name='usersettings', 20 | name='public_dir', 21 | field=models.CharField(max_length=2048, null=True), 22 | ), 23 | ] 24 | -------------------------------------------------------------------------------- /pages/migrations/0008_auto_20180726_1216.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-26 12:16 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0007_auto_20180723_0944'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='usersettings', 15 | name='save_pdf', 16 | field=models.BooleanField(default=False), 17 | ), 18 | migrations.AddField( 19 | model_name='usersettings', 20 | name='save_png', 21 | field=models.BooleanField(default=False), 22 | ), 23 | ] 24 | -------------------------------------------------------------------------------- /pages/migrations/0009_usersettings_png_quality.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-26 13:53 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0008_auto_20180726_1216'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='usersettings', 15 | name='png_quality', 16 | field=models.PositiveSmallIntegerField(default=85), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0010_usersettings_auto_archieve.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-07-31 15:52 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0009_usersettings_png_quality'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='usersettings', 15 | name='auto_archieve', 16 | field=models.BooleanField(default=False), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0011_library_tags.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-08-02 04:30 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0010_usersettings_auto_archieve'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='library', 15 | name='tags', 16 | field=models.CharField(max_length=4096, null=True), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0012_library_icon_url.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-08-04 17:47 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0011_library_tags'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='library', 15 | name='icon_url', 16 | field=models.CharField(max_length=4096, null=True), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0013_auto_20180805_1521.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-08-05 15:21 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0012_library_icon_url'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AlterField( 14 | model_name='library', 15 | name='timestamp', 16 | field=models.DateTimeField(null=True), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0014_auto_20180809_1307.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.0.7 on 2018-08-09 13:07 2 | 3 | from django.db import migrations 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0013_auto_20180805_1521'), 10 | ] 11 | 12 | operations = [ 13 | migrations.RenameField( 14 | model_name='usersettings', 15 | old_name='auto_archieve', 16 | new_name='auto_archive', 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0015_usersettings_pagination_value.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.1 on 2018-08-17 09:42 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0014_auto_20180809_1307'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='usersettings', 15 | name='pagination_value', 16 | field=models.PositiveSmallIntegerField(default=100), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0016_auto_20180925_1749.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.1 on 2018-09-25 17:49 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0015_usersettings_pagination_value'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='library', 15 | name='media_element', 16 | field=models.BooleanField(default=False), 17 | ), 18 | migrations.AddField( 19 | model_name='usersettings', 20 | name='download_manager', 21 | field=models.CharField(default='wget {iurl} -O {output}', max_length=8192), 22 | ), 23 | migrations.AddField( 24 | model_name='usersettings', 25 | name='media_streaming', 26 | field=models.BooleanField(default=False), 27 | ), 28 | ] 29 | -------------------------------------------------------------------------------- /pages/migrations/0017_library_reader_mode.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.1.2 on 2018-10-11 05:07 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0016_auto_20180925_1749'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='library', 15 | name='reader_mode', 16 | field=models.PositiveSmallIntegerField(choices=[(0, 'default'), (1, 'dark'), (2, 'light')], default=0), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0018_auto_20181011_0532.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.1.2 on 2018-10-11 05:32 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0017_library_reader_mode'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AlterField( 14 | model_name='library', 15 | name='reader_mode', 16 | field=models.PositiveSmallIntegerField(choices=[(0, 'default'), (1, 'dark'), (2, 'light'), (3, 'gray')], default=0), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0019_usersettings_reader_theme.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.1.2 on 2018-10-11 07:25 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0018_auto_20181011_0532'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='usersettings', 15 | name='reader_theme', 16 | field=models.PositiveSmallIntegerField(choices=[(0, 'default'), (1, 'dark'), (2, 'light'), (3, 'gray')], default=0), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0020_library_subdir.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.1.2 on 2018-12-08 15:28 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0019_usersettings_reader_theme'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AddField( 14 | model_name='library', 15 | name='subdir', 16 | field=models.CharField(max_length=8192, null=True), 17 | ), 18 | ] 19 | -------------------------------------------------------------------------------- /pages/migrations/0021_auto_20181208_1553.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.1.2 on 2018-12-08 15:53 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0020_library_subdir'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AlterField( 14 | model_name='library', 15 | name='icon_url', 16 | field=models.CharField(blank=True, max_length=4096, null=True), 17 | ), 18 | migrations.AlterField( 19 | model_name='library', 20 | name='media_path', 21 | field=models.CharField(blank=True, max_length=4096, null=True), 22 | ), 23 | migrations.AlterField( 24 | model_name='library', 25 | name='subdir', 26 | field=models.CharField(blank=True, max_length=8192, null=True), 27 | ), 28 | migrations.AlterField( 29 | model_name='library', 30 | name='summary', 31 | field=models.TextField(blank=True, null=True), 32 | ), 33 | migrations.AlterField( 34 | model_name='library', 35 | name='tags', 36 | field=models.CharField(blank=True, max_length=4096, null=True), 37 | ), 38 | migrations.AlterField( 39 | model_name='library', 40 | name='timestamp', 41 | field=models.DateTimeField(blank=True, null=True), 42 | ), 43 | migrations.AlterField( 44 | model_name='library', 45 | name='title', 46 | field=models.CharField(blank=True, max_length=2048, null=True), 47 | ), 48 | migrations.AlterField( 49 | model_name='library', 50 | name='url', 51 | field=models.CharField(blank=True, max_length=4096, null=True), 52 | ), 53 | ] 54 | -------------------------------------------------------------------------------- /pages/migrations/0022_auto_20181208_2051.py: -------------------------------------------------------------------------------- 1 | # Generated by Django 2.1.2 on 2018-12-08 20:51 2 | 3 | from django.db import migrations, models 4 | 5 | 6 | class Migration(migrations.Migration): 7 | 8 | dependencies = [ 9 | ('pages', '0021_auto_20181208_1553'), 10 | ] 11 | 12 | operations = [ 13 | migrations.AlterField( 14 | model_name='library', 15 | name='icon_url', 16 | field=models.CharField(max_length=4096, null=True), 17 | ), 18 | migrations.AlterField( 19 | model_name='library', 20 | name='media_path', 21 | field=models.CharField(max_length=4096, null=True), 22 | ), 23 | migrations.AlterField( 24 | model_name='library', 25 | name='subdir', 26 | field=models.CharField(max_length=8192, null=True), 27 | ), 28 | migrations.AlterField( 29 | model_name='library', 30 | name='summary', 31 | field=models.TextField(null=True), 32 | ), 33 | migrations.AlterField( 34 | model_name='library', 35 | name='tags', 36 | field=models.CharField(max_length=4096, null=True), 37 | ), 38 | migrations.AlterField( 39 | model_name='library', 40 | name='timestamp', 41 | field=models.DateTimeField(null=True), 42 | ), 43 | migrations.AlterField( 44 | model_name='library', 45 | name='title', 46 | field=models.CharField(max_length=2048, null=True), 47 | ), 48 | migrations.AlterField( 49 | model_name='library', 50 | name='url', 51 | field=models.CharField(max_length=4096, null=True), 52 | ), 53 | ] 54 | -------------------------------------------------------------------------------- /pages/migrations/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/pages/migrations/__init__.py -------------------------------------------------------------------------------- /pages/models.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of Reminiscence. 5 | 6 | Reminiscence is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Affero General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | Reminiscence is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Affero General Public License for more details. 15 | 16 | You should have received a copy of the GNU Affero General Public License 17 | along with Reminiscence. If not, see . 18 | """ 19 | 20 | from django.db import models 21 | from django.contrib.auth.models import User 22 | 23 | 24 | class UserSettings(models.Model): 25 | 26 | WHITE = 0 27 | DARK = 1 28 | LIGHT = 2 29 | GRAY = 3 30 | 31 | READER_CHOICES = ( 32 | (WHITE, 'default'), 33 | (DARK, 'dark'), 34 | (LIGHT, 'light'), 35 | (GRAY, 'gray') 36 | ) 37 | 38 | usrid = models.ForeignKey(User, related_name='usr_settings', 39 | on_delete=models.CASCADE) 40 | autotag = models.BooleanField(default=False) 41 | auto_summary = models.BooleanField(default=False) 42 | auto_archive = models.BooleanField(default=False) 43 | total_tags = models.PositiveSmallIntegerField(default=5) 44 | public_dir = models.CharField(max_length=2048, null=True) 45 | group_dir = models.CharField(max_length=2048, null=True) 46 | save_pdf = models.BooleanField(default=False) 47 | save_png = models.BooleanField(default=False) 48 | png_quality = models.PositiveSmallIntegerField(default=85) 49 | pagination_value = models.PositiveSmallIntegerField(default=100) 50 | buddy_list = models.CharField(max_length=8192, null=True) 51 | download_manager = models.CharField(max_length=8192, default='wget {iurl} -O {output}') 52 | media_streaming = models.BooleanField(default=False) 53 | reader_theme = models.PositiveSmallIntegerField(choices=READER_CHOICES, default=WHITE) 54 | 55 | def __str__(self): 56 | return self.usrid 57 | 58 | class Library(models.Model): 59 | 60 | PUBLIC = 0 61 | PRIVATE = 1 62 | GROUP = 2 63 | ACCESS_CHOICES = ( 64 | (PUBLIC, 'Public'), 65 | (PRIVATE, 'Private'), 66 | (GROUP, 'Group') 67 | ) 68 | 69 | usr = models.ForeignKey(User, related_name='usr', on_delete=models.CASCADE) 70 | directory = models.CharField(max_length=2048) 71 | url = models.CharField(max_length=4096, null=True) 72 | icon_url = models.CharField(max_length=4096, null=True) 73 | title = models.CharField(max_length=2048, null=True) 74 | timestamp = models.DateTimeField(null=True) 75 | media_path = models.CharField(max_length=4096, null=True) 76 | access = models.PositiveSmallIntegerField(choices=ACCESS_CHOICES, default=PRIVATE) 77 | summary = models.TextField(null=True) 78 | tags = models.CharField(max_length=4096, null=True) 79 | media_element = models.BooleanField(default=False) 80 | subdir = models.CharField(max_length=8192, null=True) 81 | reader_mode = models.PositiveSmallIntegerField(choices=UserSettings.READER_CHOICES, 82 | default=UserSettings.WHITE) 83 | 84 | def __str__(self): 85 | return self.usr.username 86 | 87 | 88 | class Tags(models.Model): 89 | 90 | tag = models.CharField(max_length=100, unique=True) 91 | 92 | def __str__(self): 93 | return self.tag 94 | 95 | 96 | class URLTags(models.Model): 97 | 98 | usr_id = models.ForeignKey(User, related_name='usr_tag', 99 | on_delete=models.CASCADE) 100 | url_id = models.ForeignKey(Library, 101 | related_name='url_library', 102 | on_delete=models.CASCADE) 103 | tag_id = models.ForeignKey(Tags, related_name='tag_name', 104 | on_delete=models.CASCADE) 105 | 106 | def __str__(self): 107 | return '{}, {}'.format(self.url_id, self.tag_id) 108 | 109 | 110 | class GroupTable(models.Model): 111 | 112 | user_set = models.ForeignKey(UserSettings, related_name='usr_set', 113 | on_delete=models.CASCADE) 114 | buddy = models.ForeignKey(User, related_name='usr_buddy', 115 | on_delete=models.CASCADE) 116 | 117 | -------------------------------------------------------------------------------- /pages/summarize.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of Reminiscence. 5 | 6 | Reminiscence is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Affero General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | Reminiscence is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Affero General Public License for more details. 15 | 16 | You should have received a copy of the GNU Affero General Public License 17 | along with Reminiscence. If not, see . 18 | """ 19 | 20 | import os 21 | import re 22 | import nltk 23 | from bs4 import BeautifulSoup 24 | from nltk.corpus import stopwords 25 | from nltk.stem import PorterStemmer 26 | from nltk.tokenize import word_tokenize, sent_tokenize 27 | from nltk.tag import pos_tag 28 | from django.conf import settings 29 | from collections import Counter 30 | 31 | class Summarizer: 32 | 33 | nltk_data_path = settings.NLTK_DATA_PATH 34 | nltk.data.path.append(nltk_data_path) 35 | 36 | @classmethod 37 | def check_data_path(cls): 38 | if not os.path.exists(cls.nltk_data_path): 39 | os.makedirs(cls.nltk_data_path) 40 | nltk.download( 41 | [ 42 | 'stopwords', 'punkt', 43 | 'averaged_perceptron_tagger' 44 | ], download_dir=cls.nltk_data_path 45 | ) 46 | 47 | @classmethod 48 | def get_summary_and_tags(cls, content, total_tags): 49 | cls.check_data_path() 50 | soup = BeautifulSoup(content, 'lxml') 51 | text = '' 52 | for para in soup.find_all('p'): 53 | text = text + '\n' + para.text 54 | stop_words = set(stopwords.words('english')) 55 | word_tokens = pos_tag(word_tokenize(text)) 56 | 57 | stemmer = PorterStemmer() 58 | filtered = [] 59 | 60 | for w in word_tokens: 61 | if (not w[0].lower() in stop_words and w[0].isalnum() 62 | and len(w[0]) > 2 63 | and w[1] in set(('NN', 'NNS', 'VBZ', 'NNP'))): 64 | filtered.append(w[0]) 65 | 66 | freq = Counter(filtered) 67 | tags = freq.most_common(total_tags) 68 | final_tags = [] 69 | for i, j in enumerate(tags): 70 | ps = stemmer.stem(j[0]) 71 | ntags = [stemmer.stem(l[0]) for k, l in enumerate(tags) if i != k] 72 | if ps not in ntags and ps not in stop_words: 73 | final_tags.append(j[0]) 74 | 75 | freq_dict = dict(freq) 76 | sentence = sent_tokenize(text) 77 | nd = [] 78 | words = 0 79 | for index, sen in enumerate(sentence): 80 | w = word_tokenize(sen) 81 | val = 0 82 | for j in w: 83 | val += int(freq_dict.get(j, 0)) 84 | nd.append([index, sen, val]) 85 | words += len(w) 86 | 87 | length = int(words/3) 88 | nsort = sorted(nd, key=lambda x: x[2], reverse=True) 89 | 90 | final = [] 91 | s = 0 92 | for i in nsort: 93 | w = word_tokenize(i[1]) 94 | final.append(i) 95 | s += len(w) 96 | if s > length: 97 | break 98 | 99 | final = sorted(final, key=lambda x: x[0]) 100 | 101 | sumr = '' 102 | for i, j in enumerate(final): 103 | nsum = j[1].strip() 104 | nsum = re.sub(r' +', ' ', nsum) 105 | nsum = re.sub(r'\n', '. ', nsum) 106 | if i == 0: 107 | sumr = nsum 108 | else: 109 | sumr = sumr + '\n\n' +nsum 110 | 111 | return sumr.strip(), final_tags 112 | -------------------------------------------------------------------------------- /pages/urls.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of Reminiscence. 5 | 6 | Reminiscence is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Affero General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | Reminiscence is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Affero General Public License for more details. 15 | 16 | You should have received a copy of the GNU Affero General Public License 17 | along with Reminiscence. If not, see . 18 | """ 19 | 20 | from django.urls import path 21 | from django.urls import re_path as url 22 | from .views import * 23 | from accounts.views import signup 24 | from django.contrib.auth import views as auth_views 25 | 26 | 27 | urlpatterns = [ 28 | path('', dashboard, name='home'), 29 | path('signup/', signup, name='signup'), 30 | path('annotate/annotations', create_annotations, name='ann_create'), 31 | path('annotate/annotations/', modify_annotations, name='ann_modify'), 32 | path('annotate/', annotation_root, name='ann_root'), 33 | url(r'^annotate/search', search_annotations, name='ann_s'), 34 | url(r'^settings/password/$', 35 | auth_views.PasswordChangeView.as_view(template_name='password_change.html'), 36 | name='password_change'), 37 | url(r'^settings/password/done/$', 38 | auth_views.PasswordChangeDoneView.as_view(template_name='password_change_done.html'), 39 | name='password_change_done'), 40 | url(r'^logout/', auth_views.LogoutView.as_view(), name='logout'), 41 | url(r'^login/', auth_views.LoginView.as_view(template_name='login.html'), name='login'), 42 | url(r'^(?P[\w\d.@+-]+)/?$', dashboard, name='home_page'), 43 | url(r'^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-]+)/epub-bookmark/(?P[\d]+\/(epubcfi)(.?)*)', navigate_directory, name='navigate_directory_epub'), 44 | url(r'^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-]+)/?$', navigate_directory, name='navigate_directory'), 45 | 46 | url(r'^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-]+)/(?P[\d]+)/archive/EPUBDIR/(?P(.?)*)', perform_epub_operation, name='epub_meta'), 47 | 48 | url(r'^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-]+)/(?P[\d]+)/archive/EPUBDIR/read-epub$', perform_epub_operation, name='epub_read_file'), 49 | 50 | url(r'^(?P[\w\d.@+-]+)/subdir/(?P[\w\d\s.&@+-\/]+)/(?P[\d]+)/archive/EPUBDIR/(?P(.?)*)', perform_epub_operation, name='subdir_epub_meta'), 51 | 52 | url(r'^(?P[\w\d.@+-]+)/subdir/(?P[\w\d\s.&@+-\/]+)/(?P[\d]+)/archive/EPUBDIR/read-epub', perform_epub_operation, name='subdir_epub_read_file'), 53 | 54 | url(r'^(?P[\w\d.@+-]+)/subdir/(?P[\w\d\s.&@+-\/]+)/epub-bookmark/(?P[\d]+\/(epubcfi)(.?)*)', navigate_subdir, name='navigate_subdir_epub'), 55 | 56 | url(r'^(?P[\w\d.@+-]+)/subdir/(?P[\w\d\s.&@+-\/]+)/(?P[\d]+)/(?P(.?)*\.(png|jpeg|jpg))', get_relative_resources, name='nav_subdir_resources'), 57 | url(r'^(?P[\w\d.@+-]+)/subdir/(?P[\w\d\s.&@+-\/]+)/(?P(readhtml|readcustom|readpdf))(?P(-[\d]+)+)$', record_reading_position, name='record_subdir_pos'), 58 | url(r'^(?P[\w\d.@+-]+)/subdir/(?P[\w\d\s.&@+-\/]+)$', navigate_subdir, name='navigate_subdir'), 59 | url(r'^(?P[\w\d.@+-]+)/tag/(?P[\w\d\s.&@+-]+)/?$', navigate_directory, name='navigate_tag'), 60 | path('/api/request', api_points, name='api_points'), 61 | path('/profile/public', public_profile, name='public_profile'), 62 | path('/profile/group', group_profile, name='group_profile'), 63 | url(r'^(?P[\w\d.@+-]+)/getarchivedvideo/(?P[\w\d\_\-]+)/?$', get_archived_video_link, name='get_video'), 64 | url(r'^(?P[\w\d.@+-]+)/getarchivedplaylist/(?P[\w\d\s.&@+-\/]+)/playlist/(?P[\w\d\_\-]+)/?$', get_archived_playlist, name='get_playlist'), 65 | path('//rename', rename_operation, name='rename_operation'), 66 | path('//remove', remove_operation, name='remove_operation'), 67 | path('///archive', perform_link_operation, name='archive_request'), 68 | path('///archived-note', perform_link_operation, name='archive_note_request'), 69 | path('///archived-note-save', perform_link_operation, name='archive_note_request_save'), 70 | path('///remove', perform_link_operation, name='remove_operation_link'), 71 | url(r'^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-\/]+)/(?P[\d]+)/(?P(readpdf|readcustom|readhtmtl|pdf-annotpdf))(?P(\-[\d]+)+)$', record_reading_position, name='record_pos'), 72 | path('///read', perform_link_operation, name='read_link'), 73 | path('///read-dark', perform_link_operation, name='read_dark'), 74 | path('///read-light', perform_link_operation, name='read_light'), 75 | path('///read-default', perform_link_operation, name='read_default'), 76 | path('///read-gray', perform_link_operation, name='read_gray'), 77 | path('///read-pdf', perform_link_operation, name='read_pdf'), 78 | path('///pdf-annot', perform_link_operation, name='pdf_annot'), 79 | path('///read-png', perform_link_operation, name='read_png'), 80 | path('///read-html', perform_link_operation, name='read_html'), 81 | url(r'^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-]+)/(?P[\d]+)/resources/', get_resources, name='navigate_resources'), 82 | 83 | url(r'^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-\/]+)/(?P[\d]+)/resources/', get_resources, name='navigate_resources_subdir'), 84 | path('///edit-bookmark', perform_link_operation, name='edit_bookmark'), 85 | path('///move-bookmark', perform_link_operation, name='move_bookmark'), 86 | url('^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-\/]+)/move-bookmark-multiple$', perform_link_operation, name='move_bookmark_multiple'), 87 | url('^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-\/]+)/archive-bookmark-multiple$', perform_link_operation, name='archive_bookmark_multiple'), 88 | url('^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-\/]+)/merge-bookmark-with$', perform_link_operation, name='merge_bookmark_with'), 89 | url('^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-\/]+)/edit-tags-multiple$', perform_link_operation, name='edit_tags_multiple'), 90 | url(r'^(?P[\w\d.@+-]+)/(?P[\w\d\s.&@+-\/]+)/(?P[\d]+)/(?P(.?)*)', get_relative_resources, name='navigate_url_resources') 91 | ] 92 | 93 | #url(r'^.*$', default_dest, name='catch_all') 94 | -------------------------------------------------------------------------------- /pages/utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of Reminiscence. 5 | 6 | Reminiscence is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Affero General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | Reminiscence is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Affero General Public License for more details. 15 | 16 | You should have received a copy of the GNU Affero General Public License 17 | along with Reminiscence. If not, see . 18 | """ 19 | 20 | import re 21 | import os 22 | import html 23 | import logging 24 | from .models import Library 25 | from .dbaccess import DBAccess as dbxs 26 | from datetime import datetime 27 | from django.utils import timezone 28 | from mimetypes import guess_type, guess_extension 29 | from django.conf import settings 30 | from vinanti import Vinanti 31 | 32 | logger = logging.getLogger(__name__) 33 | 34 | class ImportBookmarks: 35 | 36 | vnt = Vinanti(block=settings.VINANTI_BLOCK, 37 | hdrs={'User-Agent':settings.USER_AGENT}, 38 | max_requests=settings.VINANTI_MAX_REQUESTS, 39 | backend=settings.VINANTI_BACKEND) 40 | 41 | @classmethod 42 | def import_bookmarks(cls, usr, settings_row, import_file, mode='file'): 43 | book_dict = cls.convert_bookmark_to_dict(import_file, mode=mode) 44 | if not os.path.exists(settings.FAVICONS_STATIC): 45 | os.makedirs(settings.FAVICONS_STATIC) 46 | insert_links_list = [] 47 | insert_dir_list = [] 48 | url_list = [] 49 | for dirname in book_dict: 50 | if '/' in dirname or ':' in dirname: 51 | dirname = re.sub(r'/|:', '-', dirname) 52 | if dirname: 53 | qdir = Library.objects.filter(usr=usr, directory=dirname) 54 | if not qdir: 55 | dirlist = Library(usr=usr, directory=dirname, timestamp=timezone.now()) 56 | insert_dir_list.append(dirlist) 57 | if insert_dir_list: 58 | Library.objects.bulk_create(insert_dir_list) 59 | uqlist = Library.objects.filter(usr=usr).only('directory', 'url') 60 | urlset = set() 61 | if uqlist: 62 | urlset = set([(i.directory, i.url) for i in uqlist if i.url]) 63 | for dirname, links in book_dict.items(): 64 | for val in links: 65 | url, icon_u, add_date, title, descr = val 66 | url_tuple = (dirname, url) 67 | if url_tuple not in urlset: 68 | logger.info(val) 69 | add_date = datetime.fromtimestamp(int(add_date)) 70 | lib = Library(usr=usr, directory=dirname, url=url, 71 | icon_url=icon_u, timestamp=add_date, 72 | title=title, summary=descr) 73 | insert_links_list.append(lib) 74 | url_list.append(url) 75 | else: 76 | logger.info('{}-->{}; already exists'.format(dirname, url)) 77 | cls.insert_in_bulk(usr, settings_row, insert_links_list, url_list) 78 | 79 | @classmethod 80 | def insert_in_bulk(cls, usr, settings_row, insert_links_list, url_list): 81 | if insert_links_list: 82 | Library.objects.bulk_create(insert_links_list) 83 | 84 | qlist = Library.objects.filter(usr=usr, url__in=url_list) 85 | row_list = [] 86 | for row in qlist: 87 | icon_url = row.icon_url 88 | row_id = row.id 89 | url = row.url 90 | if url: 91 | row.media_path = cls.get_media_path(url, row_id) 92 | final_favicon_path = os.path.join(settings.FAVICONS_STATIC, str(row_id) + '.ico') 93 | row_list.append((row.icon_url, final_favicon_path)) 94 | row.save() 95 | for iurl, dest in row_list: 96 | if iurl and iurl.startswith('http'): 97 | cls.vnt.get(iurl, out=dest) 98 | 99 | if (settings_row and (settings_row.auto_archive 100 | or settings_row.auto_summary or settings_row.autotag)): 101 | for row in qlist: 102 | if row.url: 103 | dbxs.process_add_url.delay( 104 | usr.id, row.url, 105 | row.directory, 106 | archive_html=False, 107 | row_id=row.id, 108 | media_path=row.media_path 109 | ) 110 | 111 | 112 | @staticmethod 113 | def get_media_path(url, row_id): 114 | content_type = guess_type(url)[0] 115 | if content_type and content_type == 'text/plain': 116 | ext = '.txt' 117 | elif content_type: 118 | ext = guess_extension(content_type) 119 | else: 120 | ext = '.htm' 121 | out_dir = ext[1:].upper() 122 | out_title = str(row_id) + str(ext) 123 | media_dir = os.path.join(settings.ARCHIVE_LOCATION, out_dir) 124 | if not os.path.exists(media_dir): 125 | os.makedirs(media_dir) 126 | 127 | media_path_parent = os.path.join(media_dir, str(row_id)) 128 | if not os.path.exists(media_path_parent): 129 | os.makedirs(media_path_parent) 130 | 131 | media_path = os.path.join(media_path_parent, out_title) 132 | return media_path 133 | 134 | @staticmethod 135 | def convert_bookmark_to_dict(import_file, mode='file'): 136 | links_dict = {} 137 | if mode == 'file': 138 | content = "" 139 | with open(import_file, 'r', encoding='utf-8') as fd: 140 | content = fd.read() 141 | else: 142 | content = import_file 143 | if content: 144 | content = re.sub('ICON="(.*?)"', "", content) 145 | ncontent = re.sub('\n', " ", content) 146 | links_group = re.findall('
', ncontent) 147 | if not links_group: 148 | title_search = re.search('(?P<title>.*?)', ncontent) 149 | if title_search: 150 | title_bookmark = title_search.group('title') 151 | else: 152 | title_bookmark = 'My Bookmarks' 153 | logger.debug('Adding Bookmark Directory: {}'.format(title_bookmark)) 154 | ncontent = ncontent.replace('', '

{}

'.format(title_bookmark)) 155 | links_group = re.findall('
', ncontent) 156 | nsr = 0 157 | nlinks = [] 158 | for i, j in enumerate(links_group): 159 | j = j + '
' 160 | nlinks.clear() 161 | dirfield = re.search('>(?P.*?)', j) 162 | if dirfield: 163 | dirname = html.unescape(dirfield.group('dir')) 164 | else: 165 | dirname = 'Unknown' 166 | links = re.findall('A HREF="(?P.*?)"(?P.*?)
', j) 167 | for url, extra in links: 168 | dt = re.search('ADD_DATE="(?P.*?)"', extra) 169 | add_date = dt.group('add_date') 170 | dt = re.search('ICON_URI="(?P.*?)"', extra) 171 | if dt: 172 | icon_u = dt.group('icon') 173 | else: 174 | icon_u = '' 175 | dt = re.search('>(?P.*?)</A>', extra) 176 | if dt: 177 | title = html.unescape(dt.group('title')) 178 | else: 179 | title = 'No Title' 180 | dt = re.search('<DD>(?P<descr>.*?)(<DT>)?', extra) 181 | if dt: 182 | descr = html.unescape(dt.group('descr')) 183 | else: 184 | descr = 'Not Available' 185 | logger.debug(url) 186 | nlinks.append((url, icon_u, add_date, title, descr)) 187 | if dirname in links_dict: 188 | dirname = '{}-{}'.format(dirname, nsr) 189 | nsr += 1 190 | links_dict.update({dirname:nlinks.copy()}) 191 | else: 192 | logger.error('File format not recognized. Report the issue.') 193 | return links_dict 194 | 195 | 196 | #https://stackoverflow.com/questions/33208849/python-django-streaming-video-mp4-file-using-httpresponse 197 | 198 | class RangeFileResponse: 199 | 200 | def __init__(self, filelike, blksize=8192, offset=0, length=None): 201 | self.filelike = filelike 202 | self.filelike.seek(offset, os.SEEK_SET) 203 | self.remaining = length 204 | self.blksize = blksize 205 | 206 | def close(self): 207 | if hasattr(self.filelike, 'close'): 208 | self.filelike.close() 209 | 210 | def __iter__(self): 211 | return self 212 | 213 | def __next__(self): 214 | if self.remaining is None: 215 | data = self.filelike.read(self.blksize) 216 | if data: 217 | return data 218 | raise StopIteration() 219 | else: 220 | if self.remaining <= 0: 221 | raise StopIteration() 222 | data = self.filelike.read(min(self.remaining, self.blksize)) 223 | if not data: 224 | raise StopIteration() 225 | self.remaining -= len(data) 226 | return data 227 | -------------------------------------------------------------------------------- /reminiscence/__init__.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | 3 | # This will make sure the app is always imported when 4 | # Django starts so that shared_task will use this app. 5 | from .celery import app as celery_app 6 | 7 | __title__ = 'Reminiscence: Self-hosted bookmark and archive manager' 8 | __version__ = '0.3.0' 9 | __author__ = 'kanishka-linux (AAK)' 10 | __license__ = 'AGPLv3' 11 | __copyright__ = 'Copyright (C) 2018 kanishka-linux (AAK) kanishka.linux@gmail.com' 12 | -------------------------------------------------------------------------------- /reminiscence/celery.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | import os 3 | from celery import Celery 4 | from django.conf import settings 5 | 6 | # set the default Django settings module for the 'celery' program. 7 | os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'reminiscence.settings') 8 | app = Celery('reminiscence') 9 | 10 | # Using a string here means the worker will not have to 11 | # pickle the object when using Windows. 12 | app.config_from_object('django.conf:settings') 13 | app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) 14 | 15 | 16 | @app.task(bind=True) 17 | def debug_task(self): 18 | print('Request: {0!r}'.format(self.request)) 19 | -------------------------------------------------------------------------------- /reminiscence/defaultsettings.py: -------------------------------------------------------------------------------- 1 | """ 2 | Django settings for reminiscence project. 3 | 4 | Generated by 'django-admin startproject' using Django 2.0.6. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/2.0/topics/settings/ 8 | 9 | For the full list of settings and their values, see 10 | https://docs.djangoproject.com/en/2.0/ref/settings/ 11 | """ 12 | 13 | import os 14 | import re 15 | 16 | # Build paths inside the project like this: os.path.join(BASE_DIR, ...) 17 | BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 18 | 19 | 20 | # Quick-start development settings - unsuitable for production 21 | # See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/ 22 | 23 | # SECURITY WARNING: keep the secret key used in production secret! 24 | 25 | SECRET_KEY = '<Enter Secret Key Here>' 26 | 27 | # SECURITY WARNING: don't run with debug turned on in production! 28 | 29 | DEBUG = True 30 | 31 | ALLOWED_HOSTS = ['*'] 32 | 33 | 34 | LOGGING = { 35 | 'version': 1, 36 | 'disable_existing_loggers': False, 37 | 'formatters': { 38 | 'verbose': { 39 | 'format': '%(levelname)s | %(asctime)s | %(module)s | %(message)s', 40 | 'datefmt': '%m/%d/%Y %I:%M:%S %p', 41 | }, 42 | 'simple': { 43 | 'format': '%(levelname)s %(message)s' 44 | }, 45 | }, 46 | 'filters': { 47 | 'require_debug_false': { 48 | '()': 'django.utils.log.RequireDebugFalse', 49 | }, 50 | }, 51 | 'handlers': { 52 | 'file': { 53 | 'level':'DEBUG', 54 | 'class':'logging.handlers.RotatingFileHandler', 55 | 'formatter': 'verbose', 56 | 'filename': os.path.join(BASE_DIR, 'logs', 'reminiscence.log'), 57 | 'maxBytes': 1024*1024*10, 58 | 'backupCount': 5, 59 | }, 60 | }, 61 | 'loggers': { 62 | 'django.server': { 63 | 'handlers': ['file'], 64 | 'level': 'DEBUG', 65 | 'propagate': True, 66 | }, 67 | 'reminiscence': { 68 | 'handlers': ['file'], 69 | 'level': 'DEBUG', 70 | 'propagate': True, 71 | }, 72 | } 73 | } 74 | 75 | 76 | # Application definition 77 | 78 | INSTALLED_APPS = [ 79 | 'django.contrib.admin', 80 | 'django.contrib.auth', 81 | 'django.contrib.contenttypes', 82 | 'django.contrib.sessions', 83 | 'django.contrib.messages', 84 | 'django.contrib.staticfiles', 85 | 'pages', 86 | 'accounts', 87 | 'widget_tweaks', 88 | 'vinanti', 89 | 'rest_framework', 90 | 'rest_framework.authtoken', 91 | 'restapi' 92 | ] 93 | 94 | REST_FRAMEWORK = { 95 | 'DEFAULT_AUTHENTICATION_CLASSES': [ 96 | 'rest_framework.authentication.TokenAuthentication' 97 | ], 98 | } 99 | 100 | MIDDLEWARE = [ 101 | 'django.middleware.security.SecurityMiddleware', 102 | 'django.contrib.sessions.middleware.SessionMiddleware', 103 | 'django.middleware.common.CommonMiddleware', 104 | 'django.middleware.csrf.CsrfViewMiddleware', 105 | 'django.contrib.auth.middleware.AuthenticationMiddleware', 106 | 'django.contrib.messages.middleware.MessageMiddleware', 107 | 'django.middleware.clickjacking.XFrameOptionsMiddleware', 108 | ] 109 | 110 | ROOT_URLCONF = 'reminiscence.urls' 111 | 112 | # Add root url location, keep it blank or add location ex: /bookmark 113 | 114 | ROOT_URL_LOCATION = '' 115 | 116 | TEMPLATES = [ 117 | { 118 | 'BACKEND': 'django.template.backends.django.DjangoTemplates', 119 | 'DIRS': [ 120 | os.path.join(BASE_DIR, 'templates') 121 | ], 122 | 'APP_DIRS': True, 123 | 'OPTIONS': { 124 | 'context_processors': [ 125 | 'django.template.context_processors.debug', 126 | 'django.template.context_processors.request', 127 | 'django.contrib.auth.context_processors.auth', 128 | 'django.contrib.messages.context_processors.messages', 129 | ], 130 | }, 131 | }, 132 | ] 133 | 134 | WSGI_APPLICATION = 'reminiscence.wsgi.application' 135 | 136 | 137 | # Database 138 | # https://docs.djangoproject.com/en/2.0/ref/settings/#databases 139 | 140 | DATABASES = { 141 | 'default': { 142 | 'ENGINE': 'django.db.backends.sqlite3', 143 | 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), 144 | } 145 | } 146 | 147 | 148 | # Password validation 149 | # https://docs.djangoproject.com/en/2.0/ref/settings/#auth-password-validators 150 | 151 | AUTH_PASSWORD_VALIDATORS = [ 152 | { 153 | 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', 154 | }, 155 | { 156 | 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', 157 | }, 158 | { 159 | 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', 160 | }, 161 | { 162 | 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', 163 | }, 164 | ] 165 | 166 | 167 | # Internationalization 168 | # https://docs.djangoproject.com/en/2.0/topics/i18n/ 169 | 170 | LANGUAGE_CODE = 'en-us' 171 | 172 | TIME_ZONE = 'UTC' 173 | 174 | USE_I18N = True 175 | 176 | USE_L10N = True 177 | 178 | USE_TZ = True 179 | 180 | 181 | # Static files (CSS, JavaScript, Images) 182 | # https://docs.djangoproject.com/en/2.0/howto/static-files/ 183 | 184 | STATIC_URL = '/static/' 185 | 186 | STATICFILES_DIRS = [ 187 | os.path.join(BASE_DIR, 'static'), 188 | ] 189 | 190 | FAVICONS_STATIC = os.path.join(BASE_DIR, 'static', 'favicons') 191 | 192 | DEFAULT_FAVICON_PATH = os.path.join(BASE_DIR, 'static', 'archive.svg') 193 | 194 | LOGOUT_REDIRECT_URL = 'home' 195 | 196 | LOGIN_REDIRECT_URL = 'home' 197 | 198 | LOGIN_URL = 'login' 199 | 200 | RANGE_REGEX = re.compile(r'bytes\s*=\s*(\d+)\s*-\s*(\d*)', re.I) 201 | 202 | # Expiry Limit for Archived Public Media link in hours 203 | 204 | VIDEO_ID_EXPIRY_LIMIT = 24 205 | 206 | # Maximum items allowed in Public Playlist 207 | 208 | VIDEO_PUBLIC_LIST = 1000 209 | 210 | ARCHIVE_LOCATION = os.path.join(BASE_DIR, 'archive') 211 | 212 | TMP_LOCATION = os.path.join(BASE_DIR, 'tmp') 213 | 214 | USER_AGENT = 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:115.0) Gecko/20100101 Firefox/115.0' 215 | 216 | NLTK_DATA_PATH = os.path.join(BASE_DIR, 'static', 'nltk_data') 217 | 218 | USE_CELERY = True 219 | BROKER_URL = 'redis://localhost:6379' 220 | CELERY_RESULT_BACKEND = 'redis://localhost:6379' 221 | CELERY_ACCEPT_CONTENT = ['application/json'] 222 | CELERY_TASK_SERIALIZER = 'json' 223 | CELERY_RESULT_SERIALIZER = 'json' 224 | CELERY_TIMEZONE = 'UTC' 225 | 226 | USE_XVFB = False 227 | ALLOW_ANY_ONE_SIGNUP = False 228 | 229 | # Vinanti Multiprocess Settings for background tasks 230 | 231 | MULTIPROCESS_VINANTI = True 232 | MULTIPROCESS_VINANTI_MAX_REQUESTS = 4 233 | 234 | # Vinanti async HTTP client settings 235 | 236 | VINANTI_BACKEND = 'urllib' 237 | VINANTI_BLOCK = True 238 | VINANTI_MAX_REQUESTS = 20 239 | 240 | DOWNLOAD_MANAGERS_ALLOWED = ['curl', 'wget'] 241 | 242 | #Path to chromium executable or name of executable. 243 | #In some distro like ubuntu name of chromium executable is "chromium-browser". 244 | #So write it accordingly 245 | 246 | CHROMIUM_COMMAND = "chromium" 247 | 248 | CHROMIUM_SANDBOX = True 249 | 250 | DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField' 251 | -------------------------------------------------------------------------------- /reminiscence/dockersettings.py: -------------------------------------------------------------------------------- 1 | """ 2 | Django settings for reminiscence project. 3 | 4 | Generated by 'django-admin startproject' using Django 2.0.6. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/2.0/topics/settings/ 8 | 9 | For the full list of settings and their values, see 10 | https://docs.djangoproject.com/en/2.0/ref/settings/ 11 | """ 12 | 13 | import os 14 | import re 15 | 16 | # Build paths inside the project like this: os.path.join(BASE_DIR, ...) 17 | BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 18 | 19 | 20 | # Quick-start development settings - unsuitable for production 21 | # See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/ 22 | 23 | # SECURITY WARNING: keep the secret key used in production secret! 24 | 25 | SECRET_KEY = '<Enter Secret Key Here>' 26 | 27 | # SECURITY WARNING: don't run with debug turned on in production! 28 | 29 | DEBUG = False 30 | 31 | ALLOWED_HOSTS = ['*'] 32 | 33 | 34 | LOGGING = { 35 | 'version': 1, 36 | 'disable_existing_loggers': False, 37 | 'formatters': { 38 | 'verbose': { 39 | 'format': '%(levelname)s | %(asctime)s | %(module)s | %(message)s', 40 | 'datefmt': '%m/%d/%Y %I:%M:%S %p', 41 | }, 42 | 'simple': { 43 | 'format': '%(levelname)s %(message)s' 44 | }, 45 | }, 46 | 'filters': { 47 | 'require_debug_false': { 48 | '()': 'django.utils.log.RequireDebugFalse', 49 | }, 50 | }, 51 | 'handlers': { 52 | 'file': { 53 | 'level':'DEBUG', 54 | 'class':'logging.handlers.RotatingFileHandler', 55 | 'formatter': 'verbose', 56 | 'filename': os.path.join(BASE_DIR, 'logs', 'reminiscence.log'), 57 | 'maxBytes': 1024*1024*10, 58 | 'backupCount': 5, 59 | }, 60 | }, 61 | 'loggers': { 62 | 'django.server': { 63 | 'handlers': ['file'], 64 | 'level': 'DEBUG', 65 | 'propagate': True, 66 | }, 67 | 'reminiscence': { 68 | 'handlers': ['file'], 69 | 'level': 'DEBUG', 70 | 'propagate': True, 71 | }, 72 | } 73 | } 74 | 75 | 76 | # Application definition 77 | 78 | INSTALLED_APPS = [ 79 | 'django.contrib.admin', 80 | 'django.contrib.auth', 81 | 'django.contrib.contenttypes', 82 | 'django.contrib.sessions', 83 | 'django.contrib.messages', 84 | 'django.contrib.staticfiles', 85 | 'pages', 86 | 'accounts', 87 | 'widget_tweaks', 88 | 'vinanti', 89 | 'rest_framework', 90 | 'rest_framework.authtoken', 91 | 'restapi' 92 | ] 93 | 94 | REST_FRAMEWORK = { 95 | 'DEFAULT_AUTHENTICATION_CLASSES': [ 96 | 'rest_framework.authentication.TokenAuthentication' 97 | ], 98 | } 99 | 100 | MIDDLEWARE = [ 101 | 'django.middleware.security.SecurityMiddleware', 102 | 'django.contrib.sessions.middleware.SessionMiddleware', 103 | 'django.middleware.common.CommonMiddleware', 104 | 'django.middleware.csrf.CsrfViewMiddleware', 105 | 'django.contrib.auth.middleware.AuthenticationMiddleware', 106 | 'django.contrib.messages.middleware.MessageMiddleware', 107 | 'django.middleware.clickjacking.XFrameOptionsMiddleware', 108 | ] 109 | 110 | ROOT_URLCONF = 'reminiscence.urls' 111 | 112 | # Add root url location, keep it blank or add location ex: /bookmark 113 | 114 | ROOT_URL_LOCATION = '' 115 | 116 | TEMPLATES = [ 117 | { 118 | 'BACKEND': 'django.template.backends.django.DjangoTemplates', 119 | 'DIRS': [ 120 | os.path.join(BASE_DIR, 'templates') 121 | ], 122 | 'APP_DIRS': True, 123 | 'OPTIONS': { 124 | 'context_processors': [ 125 | 'django.template.context_processors.debug', 126 | 'django.template.context_processors.request', 127 | 'django.contrib.auth.context_processors.auth', 128 | 'django.contrib.messages.context_processors.messages', 129 | ], 130 | }, 131 | }, 132 | ] 133 | 134 | WSGI_APPLICATION = 'reminiscence.wsgi.application' 135 | 136 | 137 | # Database 138 | # https://docs.djangoproject.com/en/2.0/ref/settings/#databases 139 | 140 | DATABASES = { 141 | 'default': { 142 | 'ENGINE': 'django.db.backends.postgresql', 143 | 'NAME': os.environ.get('POSTGRES_NAME','postgres'), 144 | 'USER': os.environ.get('POSTGRES__USER','postgres'), 145 | 'PASSWORD': os.environ.get('POSTGRES_PASSWORD','password'), 146 | 'HOST': os.environ.get('DB_HOST','db'), 147 | 'PORT': os.environ.get('DB_PORT',5432), 148 | } 149 | } 150 | 151 | # Password validation 152 | # https://docs.djangoproject.com/en/2.0/ref/settings/#auth-password-validators 153 | 154 | AUTH_PASSWORD_VALIDATORS = [ 155 | { 156 | 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', 157 | }, 158 | { 159 | 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', 160 | }, 161 | { 162 | 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', 163 | }, 164 | { 165 | 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', 166 | }, 167 | ] 168 | 169 | 170 | # Internationalization 171 | # https://docs.djangoproject.com/en/2.0/topics/i18n/ 172 | 173 | LANGUAGE_CODE = 'en-us' 174 | 175 | TIME_ZONE = 'UTC' 176 | 177 | USE_I18N = True 178 | 179 | USE_L10N = True 180 | 181 | USE_TZ = True 182 | 183 | 184 | # Static files (CSS, JavaScript, Images) 185 | # https://docs.djangoproject.com/en/2.0/howto/static-files/ 186 | 187 | STATIC_URL = '/static/' 188 | 189 | STATIC_ROOT = os.path.join(BASE_DIR, 'static') 190 | 191 | FAVICONS_STATIC = os.path.join(BASE_DIR, 'static', 'favicons') 192 | 193 | DEFAULT_FAVICON_PATH = os.path.join(BASE_DIR, 'static', 'archive.svg') 194 | 195 | LOGOUT_REDIRECT_URL = 'home' 196 | 197 | LOGIN_REDIRECT_URL = 'home' 198 | 199 | LOGIN_URL = 'login' 200 | 201 | RANGE_REGEX = re.compile(r'bytes\s*=\s*(\d+)\s*-\s*(\d*)', re.I) 202 | 203 | # Expiry Limit for Archived Public Media link in hours 204 | 205 | VIDEO_ID_EXPIRY_LIMIT = 24 206 | 207 | # Maximum items allowed in Public Playlist 208 | 209 | VIDEO_PUBLIC_LIST = 1000 210 | 211 | ARCHIVE_LOCATION = os.path.join(BASE_DIR, 'archive') 212 | 213 | TMP_LOCATION = os.path.join(BASE_DIR, 'tmp') 214 | 215 | USER_AGENT = 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:115.0) Gecko/20100101 Firefox/115.0' 216 | 217 | NLTK_DATA_PATH = os.path.join(BASE_DIR, 'static', 'nltk_data') 218 | 219 | USE_CELERY = True 220 | BROKER_URL = 'redis://redis:6379/0' 221 | CELERY_BROKER_URL = 'redis://redis:6379/0' 222 | CELERY_RESULT_BACKEND = 'redis://redis:6379/0' 223 | CELERY_ACCEPT_CONTENT = ['application/json'] 224 | CELERY_TASK_SERIALIZER = 'json' 225 | CELERY_RESULT_SERIALIZER = 'json' 226 | CELERY_TIMEZONE = 'UTC' 227 | 228 | USE_XVFB = False 229 | ALLOW_ANY_ONE_SIGNUP = False 230 | 231 | # Vinanti Multiprocess Settings for background tasks 232 | 233 | MULTIPROCESS_VINANTI = True 234 | MULTIPROCESS_VINANTI_MAX_REQUESTS = 4 235 | 236 | # Vinanti async HTTP client settings 237 | 238 | VINANTI_BACKEND = 'urllib' 239 | VINANTI_BLOCK = True 240 | VINANTI_MAX_REQUESTS = 50 241 | 242 | DOWNLOAD_MANAGERS_ALLOWED = ['curl', 'wget'] 243 | 244 | #Path to chromium executable or name of executable. 245 | #In some distro like ubuntu name of chromium executable is "chromium-browser". 246 | #So write it accordingly 247 | 248 | CHROMIUM_COMMAND = "chromium" 249 | 250 | CHROMIUM_SANDBOX = False 251 | 252 | DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField' 253 | -------------------------------------------------------------------------------- /reminiscence/settings.py: -------------------------------------------------------------------------------- 1 | """ 2 | Django settings for reminiscence project. 3 | 4 | Generated by 'django-admin startproject' using Django 2.0.6. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/2.0/topics/settings/ 8 | 9 | For the full list of settings and their values, see 10 | https://docs.djangoproject.com/en/2.0/ref/settings/ 11 | """ 12 | 13 | import os 14 | import re 15 | 16 | # Build paths inside the project like this: os.path.join(BASE_DIR, ...) 17 | BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 18 | 19 | 20 | # Quick-start development settings - unsuitable for production 21 | # See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/ 22 | 23 | # SECURITY WARNING: keep the secret key used in production secret! 24 | 25 | SECRET_KEY = '<Enter Secret Key Here>' 26 | 27 | # SECURITY WARNING: don't run with debug turned on in production! 28 | 29 | DEBUG = True 30 | 31 | ALLOWED_HOSTS = ['*'] 32 | 33 | LOGGING = { 34 | 'version': 1, 35 | 'disable_existing_loggers': False, 36 | 'formatters': { 37 | 'verbose': { 38 | 'format': '%(levelname)s | %(asctime)s | %(module)s | %(message)s', 39 | 'datefmt': '%m/%d/%Y %I:%M:%S %p', 40 | }, 41 | 'simple': { 42 | 'format': '%(levelname)s %(message)s' 43 | }, 44 | }, 45 | 'filters': { 46 | 'require_debug_false': { 47 | '()': 'django.utils.log.RequireDebugFalse', 48 | }, 49 | }, 50 | 'handlers': { 51 | 'file': { 52 | 'level':'DEBUG', 53 | 'class':'logging.handlers.RotatingFileHandler', 54 | 'formatter': 'verbose', 55 | 'filename': os.path.join(BASE_DIR, 'logs', 'reminiscence.log'), 56 | 'maxBytes': 1024*1024*10, 57 | 'backupCount': 5, 58 | }, 59 | }, 60 | 'loggers': { 61 | 'django.server': { 62 | 'handlers': ['file'], 63 | 'level': 'DEBUG', 64 | 'propagate': True, 65 | }, 66 | 'reminiscence': { 67 | 'handlers': ['file'], 68 | 'level': 'DEBUG', 69 | 'propagate': True, 70 | }, 71 | } 72 | } 73 | 74 | 75 | # Application definition 76 | 77 | INSTALLED_APPS = [ 78 | 'django.contrib.admin', 79 | 'django.contrib.auth', 80 | 'django.contrib.contenttypes', 81 | 'django.contrib.sessions', 82 | 'django.contrib.messages', 83 | 'django.contrib.staticfiles', 84 | 'pages', 85 | 'accounts', 86 | 'widget_tweaks', 87 | 'vinanti', 88 | 'rest_framework', 89 | 'rest_framework.authtoken', 90 | 'restapi' 91 | ] 92 | 93 | REST_FRAMEWORK = { 94 | 'DEFAULT_AUTHENTICATION_CLASSES': [ 95 | 'rest_framework.authentication.TokenAuthentication' 96 | ], 97 | } 98 | 99 | MIDDLEWARE = [ 100 | 'django.middleware.security.SecurityMiddleware', 101 | 'django.contrib.sessions.middleware.SessionMiddleware', 102 | 'django.middleware.common.CommonMiddleware', 103 | 'django.middleware.csrf.CsrfViewMiddleware', 104 | 'django.contrib.auth.middleware.AuthenticationMiddleware', 105 | 'django.contrib.messages.middleware.MessageMiddleware', 106 | 'django.middleware.clickjacking.XFrameOptionsMiddleware' 107 | ] 108 | 109 | ROOT_URLCONF = 'reminiscence.urls' 110 | 111 | # Add root url location, keep it blank or add location ex: /bookmark 112 | 113 | ROOT_URL_LOCATION = '' 114 | 115 | TEMPLATES = [ 116 | { 117 | 'BACKEND': 'django.template.backends.django.DjangoTemplates', 118 | 'DIRS': [ 119 | os.path.join(BASE_DIR, 'templates') 120 | ], 121 | 'APP_DIRS': True, 122 | 'OPTIONS': { 123 | 'context_processors': [ 124 | 'django.template.context_processors.debug', 125 | 'django.template.context_processors.request', 126 | 'django.contrib.auth.context_processors.auth', 127 | 'django.contrib.messages.context_processors.messages', 128 | ], 129 | }, 130 | }, 131 | ] 132 | 133 | WSGI_APPLICATION = 'reminiscence.wsgi.application' 134 | 135 | 136 | # Database 137 | # https://docs.djangoproject.com/en/2.0/ref/settings/#databases 138 | 139 | DATABASES = { 140 | 'default': { 141 | 'ENGINE': 'django.db.backends.sqlite3', 142 | 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), 143 | } 144 | } 145 | 146 | 147 | # Password validation 148 | # https://docs.djangoproject.com/en/2.0/ref/settings/#auth-password-validators 149 | 150 | AUTH_PASSWORD_VALIDATORS = [ 151 | { 152 | 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', 153 | }, 154 | { 155 | 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', 156 | }, 157 | { 158 | 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', 159 | }, 160 | { 161 | 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', 162 | }, 163 | ] 164 | 165 | 166 | # Internationalization 167 | # https://docs.djangoproject.com/en/2.0/topics/i18n/ 168 | 169 | LANGUAGE_CODE = 'en-us' 170 | 171 | TIME_ZONE = 'UTC' 172 | 173 | USE_I18N = True 174 | 175 | USE_L10N = True 176 | 177 | USE_TZ = True 178 | 179 | 180 | # Static files (CSS, JavaScript, Images) 181 | # https://docs.djangoproject.com/en/2.0/howto/static-files/ 182 | 183 | STATIC_URL = '/static/' 184 | 185 | STATICFILES_DIRS = [ 186 | os.path.join(BASE_DIR, 'static'), 187 | ] 188 | 189 | FAVICONS_STATIC = os.path.join(BASE_DIR, 'static', 'favicons') 190 | 191 | DEFAULT_FAVICON_PATH = os.path.join(BASE_DIR, 'static', 'archive.svg') 192 | 193 | LOGOUT_REDIRECT_URL = 'home' 194 | 195 | LOGIN_REDIRECT_URL = 'home' 196 | 197 | LOGIN_URL = 'login' 198 | 199 | RANGE_REGEX = re.compile(r'bytes\s*=\s*(\d+)\s*-\s*(\d*)', re.I) 200 | 201 | # Expiry Limit for Archived Public Media link in hours 202 | 203 | VIDEO_ID_EXPIRY_LIMIT = 24 204 | 205 | # Maximum items allowed in Public Playlist 206 | 207 | VIDEO_PUBLIC_LIST = 1000 208 | 209 | ARCHIVE_LOCATION = os.path.join(BASE_DIR, 'archive') 210 | 211 | TMP_LOCATION = os.path.join(BASE_DIR, 'tmp') 212 | 213 | USER_AGENT = 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:115.0) Gecko/20100101 Firefox/115.0' 214 | 215 | NLTK_DATA_PATH = os.path.join(BASE_DIR, 'static', 'nltk_data') 216 | 217 | USE_CELERY = True 218 | BROKER_URL = 'redis://localhost:6379' 219 | CELERY_BROKER_URL = 'redis://localhost:6379' 220 | CELERY_RESULT_BACKEND = 'redis://localhost:6379' 221 | CELERY_ACCEPT_CONTENT = ['application/json'] 222 | CELERY_TASK_SERIALIZER = 'json' 223 | CELERY_RESULT_SERIALIZER = 'json' 224 | CELERY_TIMEZONE = 'UTC' 225 | 226 | USE_XVFB = False 227 | ALLOW_ANY_ONE_SIGNUP = False 228 | 229 | # Vinanti Multiprocess Settings for background tasks 230 | 231 | MULTIPROCESS_VINANTI = True 232 | MULTIPROCESS_VINANTI_MAX_REQUESTS = 4 233 | 234 | # Vinanti async HTTP client settings 235 | 236 | VINANTI_BACKEND = 'urllib' 237 | VINANTI_BLOCK = True 238 | VINANTI_MAX_REQUESTS = 20 239 | 240 | DOWNLOAD_MANAGERS_ALLOWED = ['curl', 'wget'] 241 | 242 | #Path to chromium executable or name of executable. 243 | #In some distro like ubuntu name of chromium executable is "chromium-browser". 244 | #So write it accordingly 245 | 246 | CHROMIUM_COMMAND = "chromium" 247 | 248 | CHROMIUM_SANDBOX = True 249 | 250 | DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField' 251 | -------------------------------------------------------------------------------- /reminiscence/urls.py: -------------------------------------------------------------------------------- 1 | """helloworld URL Configuration 2 | 3 | The `urlpatterns` list routes URLs to views. For more information please see: 4 | https://docs.djangoproject.com/en/2.0/topics/http/urls/ 5 | Examples: 6 | Function views 7 | 1. Add an import: from my_app import views 8 | 2. Add a URL to urlpatterns: path('', views.home, name='home') 9 | Class-based views 10 | 1. Add an import: from other_app.views import Home 11 | 2. Add a URL to urlpatterns: path('', Home.as_view(), name='home') 12 | Including another URLconf 13 | 1. Import the include() function: from django.urls import include, path 14 | 2. Add a URL to urlpatterns: path('blog/', include('blog.urls')) 15 | """ 16 | #from django.conf.urls import url 17 | from django.contrib import admin 18 | from django.urls import path, include 19 | from django.urls import re_path as url 20 | from django.conf import settings 21 | from django.contrib.staticfiles.urls import staticfiles_urlpatterns 22 | 23 | if settings.ROOT_URL_LOCATION: 24 | root_loc = settings.ROOT_URL_LOCATION 25 | if root_loc.startswith('/'): 26 | root_loc = root_loc[1:] 27 | if not root_loc.endswith('/'): 28 | root_loc = root_loc + '/' 29 | root_loc = '^' + root_loc 30 | custom_loc = root_loc 31 | else: 32 | root_loc = '' 33 | custom_loc = '^' 34 | 35 | urlpatterns = [ 36 | url(r'{}admin/'.format(custom_loc), admin.site.urls), 37 | url(r'{}restapi/'.format(custom_loc), include('restapi.urls')), 38 | url(r'{}'.format(root_loc), include('pages.urls')), 39 | ] 40 | 41 | urlpatterns += staticfiles_urlpatterns() 42 | -------------------------------------------------------------------------------- /reminiscence/wsgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | WSGI config for helloworld project. 3 | 4 | It exposes the WSGI callable as a module-level variable named ``application``. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/2.0/howto/deployment/wsgi/ 8 | """ 9 | 10 | import os 11 | 12 | from django.core.wsgi import get_wsgi_application 13 | 14 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "helloworld.settings") 15 | 16 | application = get_wsgi_application() 17 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | aiohttp==3.9.1 2 | aiosignal==1.3.1 3 | amqp==5.2.0 4 | asgiref==3.7.2 5 | async-timeout==4.0.3 6 | attrs==23.1.0 7 | beautifulsoup4==4.12.2 8 | billiard==4.2.0 9 | bs4==0.0.1 10 | celery==5.3.6 11 | chardet==5.2.0 12 | click==8.1.7 13 | click-didyoumean==0.3.0 14 | click-plugins==1.1.1 15 | click-repl==0.3.0 16 | cssselect==1.2.0 17 | Django==4.2.8 18 | django-widget-tweaks==1.5.0 19 | djangorestframework==3.14.0 20 | frozenlist==1.4.1 21 | gunicorn==21.2.0 22 | idna==3.6 23 | joblib==1.3.2 24 | kombu==5.3.4 25 | lxml==4.9.4 26 | multidict==6.0.4 27 | nltk==3.8.1 28 | packaging==23.2 29 | prompt-toolkit==3.0.43 30 | psycopg2-binary==2.9.9 31 | python-dateutil==2.8.2 32 | pytz==2023.3.post1 33 | readability-lxml==0.8.1 34 | redis==5.0.1 35 | regex==2023.12.25 36 | six==1.16.0 37 | soupsieve==2.5 38 | sqlparse==0.4.4 39 | tomli==2.0.1 40 | tqdm==4.66.1 41 | typing_extensions==4.9.0 42 | tzdata==2023.3 43 | vine==5.1.0 44 | wcwidth==0.2.12 45 | yarl==1.9.4 46 | -------------------------------------------------------------------------------- /restapi/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/restapi/__init__.py -------------------------------------------------------------------------------- /restapi/admin.py: -------------------------------------------------------------------------------- 1 | from django.contrib import admin 2 | 3 | # Register your models here. 4 | -------------------------------------------------------------------------------- /restapi/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | 3 | 4 | class RestapiConfig(AppConfig): 5 | name = 'restapi' 6 | -------------------------------------------------------------------------------- /restapi/migrations/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/restapi/migrations/__init__.py -------------------------------------------------------------------------------- /restapi/models.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | 3 | # Create your models here. 4 | -------------------------------------------------------------------------------- /restapi/tests.py: -------------------------------------------------------------------------------- 1 | from django.test import TestCase 2 | 3 | # Create your tests here. 4 | -------------------------------------------------------------------------------- /restapi/urls.py: -------------------------------------------------------------------------------- 1 | from django.urls import path 2 | from rest_framework.authtoken.views import obtain_auth_token 3 | from . import views 4 | 5 | urlpatterns = [ 6 | path('add-url/', views.AddURL.as_view(), name='add_url'), 7 | path('list-directories/', views.ListDirectories.as_view(), name='list_directories'), 8 | path('list-added-urls/', views.ListURL.as_view(), name='list_added_urls'), 9 | path('login/', obtain_auth_token, name='get_auth_token'), 10 | path('logout/', views.Logout.as_view(), name='delete_auth_token'), 11 | ] 12 | -------------------------------------------------------------------------------- /restapi/views.py: -------------------------------------------------------------------------------- 1 | import re 2 | from collections import Counter 3 | from django.utils import timezone 4 | from rest_framework.permissions import IsAuthenticated 5 | from rest_framework.views import APIView 6 | from rest_framework.response import Response 7 | from pages.dbaccess import DBAccess as dbxs 8 | from pages.models import Library, UserSettings 9 | 10 | 11 | class AddURL(APIView): 12 | 13 | permission_classes = (IsAuthenticated,) 14 | 15 | def post(self, request): 16 | usr = request.user 17 | url = request.POST.get("url") 18 | media_link = request.POST.get("media_link") 19 | if media_link and media_link == "yes": 20 | is_media_link = True 21 | else: 22 | is_media_link = False 23 | directory = request.POST.get("directory") 24 | if directory and directory.startswith("/"): 25 | directory = directory[1:] 26 | save_favicon = request.POST.get("save_favicon") 27 | if save_favicon and save_favicon == "no": 28 | save_favicon = False 29 | else: 30 | save_favicon = True 31 | user_settings = UserSettings.objects.filter(usrid=request.user) 32 | if url: 33 | http = re.match(r'^(?:http)s?://', url) 34 | else: 35 | http = None 36 | 37 | if http and directory: 38 | if self.check_dir_and_subdir(usr, directory): 39 | dbxs.add_new_url( 40 | usr, request, 41 | directory, user_settings, 42 | is_media_link=is_media_link, 43 | url_name=url, save_favicon=save_favicon 44 | ) 45 | content = { 46 | "url": url, 47 | "is_media_link": is_media_link, 48 | "directory": directory, 49 | "status": "added" 50 | } 51 | else: 52 | content = {"msg": "Maybe required directory not found. So please create directories before adding url"} 53 | else: 54 | content = {"msg": "wrong url format or directory"} 55 | 56 | return Response(content) 57 | 58 | def check_dir_and_subdir(self, usr, dirname): 59 | if dirname.startswith("/"): 60 | dirname = dirname[1:] 61 | if '/' in dirname: 62 | pdir, subdir = dirname.rsplit('/', 1) 63 | if self.verify_parent_directory(usr, pdir): 64 | self.verify_or_create_subdirectory(usr, pdir, subdir) 65 | return True 66 | else: 67 | return False 68 | else: 69 | self.verify_or_create_parent_directory(usr, dirname) 70 | return True 71 | 72 | def verify_or_create_subdirectory(self, usr, pdir, subdir): 73 | if pdir and subdir: 74 | dirname = re.sub(r'/|:|#|\?|\\\\|\%', '-', subdir) 75 | if dirname: 76 | dirname = pdir+'/'+dirname 77 | qdir = Library.objects.filter(usr=usr, directory=dirname) 78 | if not qdir: 79 | Library.objects.create( 80 | usr=usr, 81 | directory=dirname, 82 | timestamp=timezone.now() 83 | ).save() 84 | qlist = Library.objects.filter( 85 | usr=usr, directory=pdir, 86 | url__isnull=True 87 | ).first() 88 | if qlist: 89 | if qlist.subdir: 90 | slist = qlist.subdir.split('/') 91 | if subdir not in slist: 92 | qlist.subdir = '/'.join(slist + [subdir]) 93 | qlist.save() 94 | else: 95 | qlist.subdir = subdir 96 | qlist.save() 97 | 98 | def verify_parent_directory(self, usr, dirname): 99 | qdir = Library.objects.filter(usr=usr, directory=dirname) 100 | if not qdir: 101 | return False 102 | else: 103 | return True 104 | 105 | def verify_or_create_parent_directory(self, usr, dirname): 106 | dirname = re.sub(r'/|:|#|\?|\\\\|\%', '-', dirname) 107 | if dirname and not self.verify_parent_directory(usr, dirname): 108 | Library.objects.create( 109 | usr=usr, directory=dirname, 110 | timestamp=timezone.now() 111 | ).save() 112 | 113 | 114 | class ListDirectories(APIView): 115 | 116 | permission_classes = (IsAuthenticated,) 117 | 118 | def get(self, request, format=None): 119 | usr_list = Library.objects.filter( 120 | usr=request.user 121 | ).only('directory').order_by('directory') 122 | usr_list = [i.directory for i in usr_list if i.directory and i.url] 123 | usr_list = Counter(usr_list) 124 | return Response(usr_list) 125 | 126 | 127 | class ListURL(APIView): 128 | 129 | permission_classes = (IsAuthenticated,) 130 | 131 | def post(self, request, format=None): 132 | usr = request.user 133 | dirname = request.POST.get("directory") 134 | if dirname and dirname.startswith("/"): 135 | dirname = dirname[1:] 136 | if dirname: 137 | usr_list = dbxs.get_rows_by_directory(usr, directory=dirname) 138 | nlist = dbxs.populate_usr_list( 139 | usr, usr_list, 140 | create_dict=True, short_dict=True 141 | ) 142 | return Response(nlist) 143 | else: 144 | return Response({"msg": "invalid directory"}) 145 | 146 | 147 | class Logout(APIView): 148 | 149 | permission_classes = (IsAuthenticated,) 150 | 151 | def get(self, request, format=None): 152 | request.user.auth_token.delete() 153 | return Response(status=200) 154 | -------------------------------------------------------------------------------- /static/archive.svg: -------------------------------------------------------------------------------- 1 | <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-archive"><polyline points="21 8 21 21 3 21 3 8"></polyline><rect x="1" y="3" width="22" height="5"></rect><line x1="10" y1="12" x2="14" y2="12"></line></svg> -------------------------------------------------------------------------------- /static/css/accounts.css: -------------------------------------------------------------------------------- 1 | body { 2 | background-image: url(../img/full-bloom.png); 3 | } 4 | 5 | .logo { 6 | font-family: 'Peralta', cursive; 7 | } 8 | 9 | .logo a { 10 | color: rgba(0,0,0,.9); 11 | } 12 | 13 | .logo a:hover, 14 | .logo a:active { 15 | text-decoration: none; 16 | } 17 | -------------------------------------------------------------------------------- /static/css/summernote-bs4.css: -------------------------------------------------------------------------------- 1 | @font-face{font-family:"summernote";font-style:normal;font-weight:normal;src:url("./font/summernote.eot?4c7e83314b68cfa6a0d18a8b4690044b");src:url("./font/summernote.eot?4c7e83314b68cfa6a0d18a8b4690044b#iefix") format("embedded-opentype"),url("./font/summernote.woff?4c7e83314b68cfa6a0d18a8b4690044b") format("woff"),url("./font/summernote.ttf?4c7e83314b68cfa6a0d18a8b4690044b") format("truetype")}[class^="note-icon-"]:before,[class*=" note-icon-"]:before{display:inline-block;font:normal normal normal 14px summernote;font-size:inherit;-webkit-font-smoothing:antialiased;text-decoration:inherit;text-rendering:auto;text-transform:none;vertical-align:middle;speak:none;-moz-osx-font-smoothing:grayscale}.note-icon-align-center:before,.note-icon-align-indent:before,.note-icon-align-justify:before,.note-icon-align-left:before,.note-icon-align-outdent:before,.note-icon-align-right:before,.note-icon-align:before,.note-icon-arrow-circle-down:before,.note-icon-arrow-circle-left:before,.note-icon-arrow-circle-right:before,.note-icon-arrow-circle-up:before,.note-icon-arrows-alt:before,.note-icon-arrows-h:before,.note-icon-arrows-v:before,.note-icon-bold:before,.note-icon-caret:before,.note-icon-chain-broken:before,.note-icon-circle:before,.note-icon-close:before,.note-icon-code:before,.note-icon-col-after:before,.note-icon-col-before:before,.note-icon-col-remove:before,.note-icon-eraser:before,.note-icon-font:before,.note-icon-frame:before,.note-icon-italic:before,.note-icon-link:before,.note-icon-magic:before,.note-icon-menu-check:before,.note-icon-minus:before,.note-icon-orderedlist:before,.note-icon-pencil:before,.note-icon-picture:before,.note-icon-question:before,.note-icon-redo:before,.note-icon-row-above:before,.note-icon-row-below:before,.note-icon-row-remove:before,.note-icon-special-character:before,.note-icon-square:before,.note-icon-strikethrough:before,.note-icon-subscript:before,.note-icon-summernote:before,.note-icon-superscript:before,.note-icon-table:before,.note-icon-text-height:before,.note-icon-trash:before,.note-icon-underline:before,.note-icon-undo:before,.note-icon-unorderedlist:before,.note-icon-video:before{display:inline-block;font-family:"summernote";font-style:normal;font-weight:normal;text-decoration:inherit}.note-icon-align-center:before{content:"\f101"}.note-icon-align-indent:before{content:"\f102"}.note-icon-align-justify:before{content:"\f103"}.note-icon-align-left:before{content:"\f104"}.note-icon-align-outdent:before{content:"\f105"}.note-icon-align-right:before{content:"\f106"}.note-icon-align:before{content:"\f107"}.note-icon-arrow-circle-down:before{content:"\f108"}.note-icon-arrow-circle-left:before{content:"\f109"}.note-icon-arrow-circle-right:before{content:"\f10a"}.note-icon-arrow-circle-up:before{content:"\f10b"}.note-icon-arrows-alt:before{content:"\f10c"}.note-icon-arrows-h:before{content:"\f10d"}.note-icon-arrows-v:before{content:"\f10e"}.note-icon-bold:before{content:"\f10f"}.note-icon-caret:before{content:"\f110"}.note-icon-chain-broken:before{content:"\f111"}.note-icon-circle:before{content:"\f112"}.note-icon-close:before{content:"\f113"}.note-icon-code:before{content:"\f114"}.note-icon-col-after:before{content:"\f115"}.note-icon-col-before:before{content:"\f116"}.note-icon-col-remove:before{content:"\f117"}.note-icon-eraser:before{content:"\f118"}.note-icon-font:before{content:"\f119"}.note-icon-frame:before{content:"\f11a"}.note-icon-italic:before{content:"\f11b"}.note-icon-link:before{content:"\f11c"}.note-icon-magic:before{content:"\f11d"}.note-icon-menu-check:before{content:"\f11e"}.note-icon-minus:before{content:"\f11f"}.note-icon-orderedlist:before{content:"\f120"}.note-icon-pencil:before{content:"\f121"}.note-icon-picture:before{content:"\f122"}.note-icon-question:before{content:"\f123"}.note-icon-redo:before{content:"\f124"}.note-icon-row-above:before{content:"\f125"}.note-icon-row-below:before{content:"\f126"}.note-icon-row-remove:before{content:"\f127"}.note-icon-special-character:before{content:"\f128"}.note-icon-square:before{content:"\f129"}.note-icon-strikethrough:before{content:"\f12a"}.note-icon-subscript:before{content:"\f12b"}.note-icon-summernote:before{content:"\f12c"}.note-icon-superscript:before{content:"\f12d"}.note-icon-table:before{content:"\f12e"}.note-icon-text-height:before{content:"\f12f"}.note-icon-trash:before{content:"\f130"}.note-icon-underline:before{content:"\f131"}.note-icon-undo:before{content:"\f132"}.note-icon-unorderedlist:before{content:"\f133"}.note-icon-video:before{content:"\f134"}.note-editor{position:relative}.note-editor .note-dropzone{position:absolute;z-index:100;display:none;color:#87cefa;background-color:#fff;opacity:.95}.note-editor .note-dropzone .note-dropzone-message{display:table-cell;font-size:28px;font-weight:700;text-align:center;vertical-align:middle}.note-editor .note-dropzone.hover{color:#098ddf}.note-editor.dragover .note-dropzone{display:table}.note-editor .note-editing-area{position:relative}.note-editor .note-editing-area .note-editable{outline:0}.note-editor .note-editing-area .note-editable sup{vertical-align:super}.note-editor .note-editing-area .note-editable sub{vertical-align:sub}.note-editor .note-editing-area img.note-float-left{margin-right:10px}.note-editor .note-editing-area img.note-float-right{margin-left:10px}.note-editor.note-frame{border:1px solid #a9a9a9}.note-editor.note-frame.codeview .note-editing-area .note-editable{display:none}.note-editor.note-frame.codeview .note-editing-area .note-codable{display:block}.note-editor.note-frame .note-editing-area{overflow:hidden}.note-editor.note-frame .note-editing-area .note-editable{padding:10px;overflow:auto;color:#000;word-wrap:break-word;background-color:#fff}.note-editor.note-frame .note-editing-area .note-editable[contenteditable="false"]{background-color:#e5e5e5}.note-editor.note-frame .note-editing-area .note-codable{display:none;width:100%;padding:10px;margin-bottom:0;font-family:Menlo,Monaco,monospace,sans-serif;font-size:14px;color:#ccc;background-color:#222;border:0;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0;box-shadow:none;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;-ms-box-sizing:border-box;box-sizing:border-box;resize:none}.note-editor.note-frame.fullscreen{position:fixed;top:0;left:0;z-index:1050;width:100%!important}.note-editor.note-frame.fullscreen .note-editable{background-color:#fff}.note-editor.note-frame.fullscreen .note-resizebar{display:none}.note-editor.note-frame .note-status-output{display:block;width:100%;height:20px;margin-bottom:0;font-size:14px;line-height:1.42857143;color:#000;border:0;border-top:1px solid #e2e2e2}.note-editor.note-frame .note-status-output:empty{height:0;border-top:0 solid transparent}.note-editor.note-frame .note-status-output .pull-right{float:right!important}.note-editor.note-frame .note-status-output .text-muted{color:#777}.note-editor.note-frame .note-status-output .text-primary{color:#286090}.note-editor.note-frame .note-status-output .text-success{color:#3c763d}.note-editor.note-frame .note-status-output .text-info{color:#31708f}.note-editor.note-frame .note-status-output .text-warning{color:#8a6d3b}.note-editor.note-frame .note-status-output .text-danger{color:#a94442}.note-editor.note-frame .note-status-output .alert{padding:7px 10px 2px 10px;margin:-7px 0 0 0;color:#000;background-color:#f5f5f5;border-radius:0}.note-editor.note-frame .note-status-output .alert .note-icon{margin-right:5px}.note-editor.note-frame .note-status-output .alert-success{color:#3c763d!important;background-color:#dff0d8!important}.note-editor.note-frame .note-status-output .alert-info{color:#31708f!important;background-color:#d9edf7!important}.note-editor.note-frame .note-status-output .alert-warning{color:#8a6d3b!important;background-color:#fcf8e3!important}.note-editor.note-frame .note-status-output .alert-danger{color:#a94442!important;background-color:#f2dede!important}.note-editor.note-frame .note-statusbar{background-color:#f5f5f5;border-top:1px solid #ddd;border-bottom-right-radius:4px;border-bottom-left-radius:4px}.note-editor.note-frame .note-statusbar .note-resizebar{width:100%;height:9px;padding-top:1px;cursor:ns-resize}.note-editor.note-frame .note-statusbar .note-resizebar .note-icon-bar{width:20px;margin:1px auto;border-top:1px solid #a9a9a9}.note-editor.note-frame .note-statusbar.locked .note-resizebar{cursor:default}.note-editor.note-frame .note-statusbar.locked .note-resizebar .note-icon-bar{display:none}.note-editor.note-frame .note-placeholder{padding:10px}.note-popover.popover{display:none;max-width:none}.note-popover.popover .popover-content a{display:inline-block;max-width:200px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;vertical-align:middle}.note-popover.popover .arrow{left:20px!important}.note-toolbar{position:relative;z-index:500}.note-popover .popover-content,.card-header.note-toolbar{padding:0 0 5px 5px;margin:0;background:#f5f5f5}.note-popover .popover-content>.btn-group,.card-header.note-toolbar>.btn-group{margin-top:5px;margin-right:5px;margin-left:0}.note-popover .popover-content .btn-group .note-table,.card-header.note-toolbar .btn-group .note-table{min-width:0;padding:5px}.note-popover .popover-content .btn-group .note-table .note-dimension-picker,.card-header.note-toolbar .btn-group .note-table .note-dimension-picker{font-size:18px}.note-popover .popover-content .btn-group .note-table .note-dimension-picker .note-dimension-picker-mousecatcher,.card-header.note-toolbar .btn-group .note-table .note-dimension-picker .note-dimension-picker-mousecatcher{position:absolute!important;z-index:3;width:10em;height:10em;cursor:pointer}.note-popover .popover-content .btn-group .note-table .note-dimension-picker .note-dimension-picker-unhighlighted,.card-header.note-toolbar .btn-group .note-table .note-dimension-picker .note-dimension-picker-unhighlighted{position:relative!important;z-index:1;width:5em;height:5em;background:url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABIAAAASAgMAAAAroGbEAAAACVBMVEUAAIj4+Pjp6ekKlAqjAAAAAXRSTlMAQObYZgAAAAFiS0dEAIgFHUgAAAAJcEhZcwAACxMAAAsTAQCanBgAAAAHdElNRQfYAR0BKhmnaJzPAAAAG0lEQVQI12NgAAOtVatWMTCohoaGUY+EmIkEAEruEzK2J7tvAAAAAElFTkSuQmCC') repeat}.note-popover .popover-content .btn-group .note-table .note-dimension-picker .note-dimension-picker-highlighted,.card-header.note-toolbar .btn-group .note-table .note-dimension-picker .note-dimension-picker-highlighted{position:absolute!important;z-index:2;width:1em;height:1em;background:url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABIAAAASAgMAAAAroGbEAAAACVBMVEUAAIjd6vvD2f9LKLW+AAAAAXRSTlMAQObYZgAAAAFiS0dEAIgFHUgAAAAJcEhZcwAACxMAAAsTAQCanBgAAAAHdElNRQfYAR0BKwNDEVT0AAAAG0lEQVQI12NgAAOtVatWMTCohoaGUY+EmIkEAEruEzK2J7tvAAAAAElFTkSuQmCC') repeat}.note-popover .popover-content .note-style .dropdown-style blockquote,.card-header.note-toolbar .note-style .dropdown-style blockquote,.note-popover .popover-content .note-style .dropdown-style pre,.card-header.note-toolbar .note-style .dropdown-style pre{padding:5px 10px;margin:0}.note-popover .popover-content .note-style .dropdown-style h1,.card-header.note-toolbar .note-style .dropdown-style h1,.note-popover .popover-content .note-style .dropdown-style h2,.card-header.note-toolbar .note-style .dropdown-style h2,.note-popover .popover-content .note-style .dropdown-style h3,.card-header.note-toolbar .note-style .dropdown-style h3,.note-popover .popover-content .note-style .dropdown-style h4,.card-header.note-toolbar .note-style .dropdown-style h4,.note-popover .popover-content .note-style .dropdown-style h5,.card-header.note-toolbar .note-style .dropdown-style h5,.note-popover .popover-content .note-style .dropdown-style h6,.card-header.note-toolbar .note-style .dropdown-style h6,.note-popover .popover-content .note-style .dropdown-style p,.card-header.note-toolbar .note-style .dropdown-style p{padding:0;margin:0}.note-popover .popover-content .note-color-all .dropdown-menu,.card-header.note-toolbar .note-color-all .dropdown-menu{min-width:337px}.note-popover .popover-content .note-color .dropdown-toggle,.card-header.note-toolbar .note-color .dropdown-toggle{width:20px;padding-left:5px}.note-popover .popover-content .note-color .dropdown-menu .note-palette,.card-header.note-toolbar .note-color .dropdown-menu .note-palette{display:inline-block;width:160px;margin:0}.note-popover .popover-content .note-color .dropdown-menu .note-palette:first-child,.card-header.note-toolbar .note-color .dropdown-menu .note-palette:first-child{margin:0 5px}.note-popover .popover-content .note-color .dropdown-menu .note-palette .note-palette-title,.card-header.note-toolbar .note-color .dropdown-menu .note-palette .note-palette-title{margin:2px 7px;font-size:12px;text-align:center;border-bottom:1px solid #eee}.note-popover .popover-content .note-color .dropdown-menu .note-palette .note-color-reset,.card-header.note-toolbar .note-color .dropdown-menu .note-palette .note-color-reset,.note-popover .popover-content .note-color .dropdown-menu .note-palette .note-color-select,.card-header.note-toolbar .note-color .dropdown-menu .note-palette .note-color-select{width:100%;padding:0 3px;margin:3px;font-size:11px;cursor:pointer;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px}.note-popover .popover-content .note-color .dropdown-menu .note-palette .note-color-row,.card-header.note-toolbar .note-color .dropdown-menu .note-palette .note-color-row{height:20px}.note-popover .popover-content .note-color .dropdown-menu .note-palette .note-color-reset:hover,.card-header.note-toolbar .note-color .dropdown-menu .note-palette .note-color-reset:hover{background:#eee}.note-popover .popover-content .note-color .dropdown-menu .note-palette .note-color-select-btn,.card-header.note-toolbar .note-color .dropdown-menu .note-palette .note-color-select-btn{display:none}.note-popover .popover-content .note-color .dropdown-menu .note-palette .note-holder-custom .note-color-btn,.card-header.note-toolbar .note-color .dropdown-menu .note-palette .note-holder-custom .note-color-btn{border:1px solid #eee}.note-popover .popover-content .note-para .dropdown-menu,.card-header.note-toolbar .note-para .dropdown-menu{min-width:216px;padding:5px}.note-popover .popover-content .note-para .dropdown-menu>div:first-child,.card-header.note-toolbar .note-para .dropdown-menu>div:first-child{margin-right:5px}.note-popover .popover-content .dropdown-menu,.card-header.note-toolbar .dropdown-menu{min-width:90px}.note-popover .popover-content .dropdown-menu.right,.card-header.note-toolbar .dropdown-menu.right{right:0;left:auto}.note-popover .popover-content .dropdown-menu.right::before,.card-header.note-toolbar .dropdown-menu.right::before{right:9px;left:auto!important}.note-popover .popover-content .dropdown-menu.right::after,.card-header.note-toolbar .dropdown-menu.right::after{right:10px;left:auto!important}.note-popover .popover-content .dropdown-menu.note-check a i,.card-header.note-toolbar .dropdown-menu.note-check a i{color:deepskyblue;visibility:hidden}.note-popover .popover-content .dropdown-menu.note-check a.checked i,.card-header.note-toolbar .dropdown-menu.note-check a.checked i{visibility:visible}.note-popover .popover-content .note-fontsize-10,.card-header.note-toolbar .note-fontsize-10{font-size:10px}.note-popover .popover-content .note-color-palette,.card-header.note-toolbar .note-color-palette{line-height:1}.note-popover .popover-content .note-color-palette div .note-color-btn,.card-header.note-toolbar .note-color-palette div .note-color-btn{width:20px;height:20px;padding:0;margin:0;border:1px solid #fff}.note-popover .popover-content .note-color-palette div .note-color-btn:hover,.card-header.note-toolbar .note-color-palette div .note-color-btn:hover{border:1px solid #000}.note-dialog>div{display:none}.note-dialog .form-group{margin-right:0;margin-left:0}.note-dialog .note-modal-form{margin:0}.note-dialog .note-image-dialog .note-dropzone{min-height:100px;margin-bottom:10px;font-size:30px;line-height:4;color:lightgray;text-align:center;border:4px dashed lightgray}@-moz-document url-prefix(){.note-image-input{height:auto}}.note-placeholder{position:absolute;display:none;color:gray}.note-handle .note-control-selection{position:absolute;display:none;border:1px solid black}.note-handle .note-control-selection>div{position:absolute}.note-handle .note-control-selection .note-control-selection-bg{width:100%;height:100%;background-color:black;-webkit-opacity:.3;-khtml-opacity:.3;-moz-opacity:.3;opacity:.3;-ms-filter:alpha(opacity=30);filter:alpha(opacity=30)}.note-handle .note-control-selection .note-control-handle{width:7px;height:7px;border:1px solid black}.note-handle .note-control-selection .note-control-holder{width:7px;height:7px;border:1px solid black}.note-handle .note-control-selection .note-control-sizing{width:7px;height:7px;background-color:white;border:1px solid black}.note-handle .note-control-selection .note-control-nw{top:-5px;left:-5px;border-right:0;border-bottom:0}.note-handle .note-control-selection .note-control-ne{top:-5px;right:-5px;border-bottom:0;border-left:none}.note-handle .note-control-selection .note-control-sw{bottom:-5px;left:-5px;border-top:0;border-right:0}.note-handle .note-control-selection .note-control-se{right:-5px;bottom:-5px;cursor:se-resize}.note-handle .note-control-selection .note-control-se.note-control-holder{cursor:default;border-top:0;border-left:none}.note-handle .note-control-selection .note-control-selection-info{right:0;bottom:0;padding:5px;margin:5px;font-size:12px;color:#fff;background-color:#000;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px;-webkit-opacity:.7;-khtml-opacity:.7;-moz-opacity:.7;opacity:.7;-ms-filter:alpha(opacity=70);filter:alpha(opacity=70)}.note-hint-popover{min-width:100px;padding:2px}.note-hint-popover .popover-content{max-height:150px;padding:3px;overflow:auto}.note-hint-popover .popover-content .note-hint-group .note-hint-item{display:block!important;padding:3px}.note-hint-popover .popover-content .note-hint-group .note-hint-item.active,.note-hint-popover .popover-content .note-hint-group .note-hint-item:hover{display:block;clear:both;font-weight:400;line-height:1.4;color:#fff;text-decoration:none;white-space:nowrap;cursor:pointer;background-color:#428bca;outline:0} -------------------------------------------------------------------------------- /static/css/text_layer_builder.css: -------------------------------------------------------------------------------- 1 | /* Copyright 2014 Mozilla Foundation 2 | * 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | .textLayer { 17 | position: absolute; 18 | left: 0; 19 | top: 0; 20 | right: 0; 21 | bottom: 0; 22 | overflow: hidden; 23 | opacity: 0.3; 24 | line-height: 1.0; 25 | } 26 | 27 | .textLayer > span { 28 | color: transparent; 29 | position: absolute; 30 | white-space: pre; 31 | cursor: text; 32 | transform-origin: 0% 0%; 33 | } 34 | 35 | .textLayer .highlight { 36 | margin: -1px; 37 | padding: 1px; 38 | 39 | background-color: rgb(180, 0, 170); 40 | border-radius: 4px; 41 | } 42 | 43 | .textLayer .highlight.begin { 44 | border-radius: 4px 0px 0px 4px; 45 | } 46 | 47 | .textLayer .highlight.end { 48 | border-radius: 0px 4px 4px 0px; 49 | } 50 | 51 | .textLayer .highlight.middle { 52 | border-radius: 0px; 53 | } 54 | 55 | .textLayer .highlight.selected { 56 | background-color: rgb(0, 100, 0); 57 | } 58 | 59 | .textLayer ::selection { background: rgb(0,0,255); } 60 | 61 | .textLayer .endOfContent { 62 | display: block; 63 | position: absolute; 64 | left: 0px; 65 | top: 100%; 66 | right: 0px; 67 | bottom: 0px; 68 | z-index: -1; 69 | cursor: default; 70 | user-select: none; 71 | } 72 | 73 | .textLayer .endOfContent.active { 74 | top: 0px; 75 | } 76 | -------------------------------------------------------------------------------- /static/external-link.svg: -------------------------------------------------------------------------------- 1 | <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-external-link"><path d="M18 13v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"></path><polyline points="15 3 21 3 21 9"></polyline><line x1="10" y1="14" x2="21" y2="3"></line></svg> -------------------------------------------------------------------------------- /static/folder.svg: -------------------------------------------------------------------------------- 1 | <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-folder"><path d="M22 19a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h5l2 3h9a2 2 0 0 1 2 2z"></path></svg> -------------------------------------------------------------------------------- /static/img/full-bloom.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/static/img/full-bloom.png -------------------------------------------------------------------------------- /static/js/bootbox.min.js: -------------------------------------------------------------------------------- 1 | /** 2 | * bootbox.js v4.4.0 3 | * 4 | * http://bootboxjs.com/license.txt 5 | */ 6 | !function(a,b){"use strict";"function"==typeof define&&define.amd?define(["jquery"],b):"object"==typeof exports?module.exports=b(require("jquery")):a.bootbox=b(a.jQuery)}(this,function a(b,c){"use strict";function d(a){var b=q[o.locale];return b?b[a]:q.en[a]}function e(a,c,d){a.stopPropagation(),a.preventDefault();var e=b.isFunction(d)&&d.call(c,a)===!1;e||c.modal("hide")}function f(a){var b,c=0;for(b in a)c++;return c}function g(a,c){var d=0;b.each(a,function(a,b){c(a,b,d++)})}function h(a){var c,d;if("object"!=typeof a)throw new Error("Please supply an object of options");if(!a.message)throw new Error("Please specify a message");return a=b.extend({},o,a),a.buttons||(a.buttons={}),c=a.buttons,d=f(c),g(c,function(a,e,f){if(b.isFunction(e)&&(e=c[a]={callback:e}),"object"!==b.type(e))throw new Error("button with key "+a+" must be an object");e.label||(e.label=a),e.className||(e.className=2>=d&&f===d-1?"btn-primary":"btn-default")}),a}function i(a,b){var c=a.length,d={};if(1>c||c>2)throw new Error("Invalid argument length");return 2===c||"string"==typeof a[0]?(d[b[0]]=a[0],d[b[1]]=a[1]):d=a[0],d}function j(a,c,d){return b.extend(!0,{},a,i(c,d))}function k(a,b,c,d){var e={className:"bootbox-"+a,buttons:l.apply(null,b)};return m(j(e,d,c),b)}function l(){for(var a={},b=0,c=arguments.length;c>b;b++){var e=arguments[b],f=e.toLowerCase(),g=e.toUpperCase();a[f]={label:d(g)}}return a}function m(a,b){var d={};return g(b,function(a,b){d[b]=!0}),g(a.buttons,function(a){if(d[a]===c)throw new Error("button key "+a+" is not allowed (options are "+b.join("\n")+")")}),a}var n={dialog:"<div class='bootbox modal' tabindex='-1' role='dialog'><div class='modal-dialog'><div class='modal-content'><div class='modal-body'><div class='bootbox-body'></div></div></div></div></div>",header:"<div class='modal-header'><h4 class='modal-title'></h4></div>",footer:"<div class='modal-footer'></div>",closeButton:"<button type='button' class='bootbox-close-button close' data-dismiss='modal' aria-hidden='true'>×</button>",form:"<form class='bootbox-form'></form>",inputs:{text:"<input class='bootbox-input bootbox-input-text form-control' autocomplete=off type=text />",textarea:"<textarea class='bootbox-input bootbox-input-textarea form-control'></textarea>",email:"<input class='bootbox-input bootbox-input-email form-control' autocomplete='off' type='email' />",select:"<select class='bootbox-input bootbox-input-select form-control'></select>",checkbox:"<div class='checkbox'><label><input class='bootbox-input bootbox-input-checkbox' type='checkbox' /></label></div>",date:"<input class='bootbox-input bootbox-input-date form-control' autocomplete=off type='date' />",time:"<input class='bootbox-input bootbox-input-time form-control' autocomplete=off type='time' />",number:"<input class='bootbox-input bootbox-input-number form-control' autocomplete=off type='number' />",password:"<input class='bootbox-input bootbox-input-password form-control' autocomplete='off' type='password' />"}},o={locale:"en",backdrop:"static",animate:!0,className:null,closeButton:!0,show:!0,container:"body"},p={};p.alert=function(){var a;if(a=k("alert",["ok"],["message","callback"],arguments),a.callback&&!b.isFunction(a.callback))throw new Error("alert requires callback property to be a function when provided");return a.buttons.ok.callback=a.onEscape=function(){return b.isFunction(a.callback)?a.callback.call(this):!0},p.dialog(a)},p.confirm=function(){var a;if(a=k("confirm",["cancel","confirm"],["message","callback"],arguments),a.buttons.cancel.callback=a.onEscape=function(){return a.callback.call(this,!1)},a.buttons.confirm.callback=function(){return a.callback.call(this,!0)},!b.isFunction(a.callback))throw new Error("confirm requires a callback");return p.dialog(a)},p.prompt=function(){var a,d,e,f,h,i,k;if(f=b(n.form),d={className:"bootbox-prompt",buttons:l("cancel","confirm"),value:"",inputType:"text"},a=m(j(d,arguments,["title","callback"]),["cancel","confirm"]),i=a.show===c?!0:a.show,a.message=f,a.buttons.cancel.callback=a.onEscape=function(){return a.callback.call(this,null)},a.buttons.confirm.callback=function(){var c;switch(a.inputType){case"text":case"textarea":case"email":case"select":case"date":case"time":case"number":case"password":c=h.val();break;case"checkbox":var d=h.find("input:checked");c=[],g(d,function(a,d){c.push(b(d).val())})}return a.callback.call(this,c)},a.show=!1,!a.title)throw new Error("prompt requires a title");if(!b.isFunction(a.callback))throw new Error("prompt requires a callback");if(!n.inputs[a.inputType])throw new Error("invalid prompt type");switch(h=b(n.inputs[a.inputType]),a.inputType){case"text":case"textarea":case"email":case"date":case"time":case"number":case"password":h.val(a.value);break;case"select":var o={};if(k=a.inputOptions||[],!b.isArray(k))throw new Error("Please pass an array of input options");if(!k.length)throw new Error("prompt with select requires options");g(k,function(a,d){var e=h;if(d.value===c||d.text===c)throw new Error("given options in wrong format");d.group&&(o[d.group]||(o[d.group]=b("<optgroup/>").attr("label",d.group)),e=o[d.group]),e.append("<option value='"+d.value+"'>"+d.text+"</option>")}),g(o,function(a,b){h.append(b)}),h.val(a.value);break;case"checkbox":var q=b.isArray(a.value)?a.value:[a.value];if(k=a.inputOptions||[],!k.length)throw new Error("prompt with checkbox requires options");if(!k[0].value||!k[0].text)throw new Error("given options in wrong format");h=b("<div/>"),g(k,function(c,d){var e=b(n.inputs[a.inputType]);e.find("input").attr("value",d.value),e.find("label").append(d.text),g(q,function(a,b){b===d.value&&e.find("input").prop("checked",!0)}),h.append(e)})}return a.placeholder&&h.attr("placeholder",a.placeholder),a.pattern&&h.attr("pattern",a.pattern),a.maxlength&&h.attr("maxlength",a.maxlength),f.append(h),f.on("submit",function(a){a.preventDefault(),a.stopPropagation(),e.find(".btn-primary").click()}),e=p.dialog(a),e.off("shown.bs.modal"),e.on("shown.bs.modal",function(){h.focus()}),i===!0&&e.modal("show"),e},p.dialog=function(a){a=h(a);var d=b(n.dialog),f=d.find(".modal-dialog"),i=d.find(".modal-body"),j=a.buttons,k="",l={onEscape:a.onEscape};if(b.fn.modal===c)throw new Error("$.fn.modal is not defined; please double check you have included the Bootstrap JavaScript library. See http://getbootstrap.com/javascript/ for more details.");if(g(j,function(a,b){k+="<button data-bb-handler='"+a+"' type='button' class='btn "+b.className+"'>"+b.label+"</button>",l[a]=b.callback}),i.find(".bootbox-body").html(a.message),a.animate===!0&&d.addClass("fade"),a.className&&d.addClass(a.className),"large"===a.size?f.addClass("modal-lg"):"small"===a.size&&f.addClass("modal-sm"),a.title&&i.before(n.header),a.closeButton){var m=b(n.closeButton);a.title?d.find(".modal-header").prepend(m):m.css("margin-top","-10px").prependTo(i)}return a.title&&d.find(".modal-title").html(a.title),k.length&&(i.after(n.footer),d.find(".modal-footer").html(k)),d.on("hidden.bs.modal",function(a){a.target===this&&d.remove()}),d.on("shown.bs.modal",function(){d.find(".btn-primary:first").focus()}),"static"!==a.backdrop&&d.on("click.dismiss.bs.modal",function(a){d.children(".modal-backdrop").length&&(a.currentTarget=d.children(".modal-backdrop").get(0)),a.target===a.currentTarget&&d.trigger("escape.close.bb")}),d.on("escape.close.bb",function(a){l.onEscape&&e(a,d,l.onEscape)}),d.on("click",".modal-footer button",function(a){var c=b(this).data("bb-handler");e(a,d,l[c])}),d.on("click",".bootbox-close-button",function(a){e(a,d,l.onEscape)}),d.on("keyup",function(a){27===a.which&&d.trigger("escape.close.bb")}),b(a.container).append(d),d.modal({backdrop:a.backdrop?"static":!1,keyboard:!1,show:!1}),a.show&&d.modal("show"),d},p.setDefaults=function(){var a={};2===arguments.length?a[arguments[0]]=arguments[1]:a=arguments[0],b.extend(o,a)},p.hideAll=function(){return b(".bootbox").modal("hide"),p};var q={bg_BG:{OK:"Ок",CANCEL:"Отказ",CONFIRM:"Потвърждавам"},br:{OK:"OK",CANCEL:"Cancelar",CONFIRM:"Sim"},cs:{OK:"OK",CANCEL:"Zrušit",CONFIRM:"Potvrdit"},da:{OK:"OK",CANCEL:"Annuller",CONFIRM:"Accepter"},de:{OK:"OK",CANCEL:"Abbrechen",CONFIRM:"Akzeptieren"},el:{OK:"Εντάξει",CANCEL:"Ακύρωση",CONFIRM:"Επιβεβαίωση"},en:{OK:"OK",CANCEL:"Cancel",CONFIRM:"OK"},es:{OK:"OK",CANCEL:"Cancelar",CONFIRM:"Aceptar"},et:{OK:"OK",CANCEL:"Katkesta",CONFIRM:"OK"},fa:{OK:"قبول",CANCEL:"لغو",CONFIRM:"تایید"},fi:{OK:"OK",CANCEL:"Peruuta",CONFIRM:"OK"},fr:{OK:"OK",CANCEL:"Annuler",CONFIRM:"D'accord"},he:{OK:"אישור",CANCEL:"ביטול",CONFIRM:"אישור"},hu:{OK:"OK",CANCEL:"Mégsem",CONFIRM:"Megerősít"},hr:{OK:"OK",CANCEL:"Odustani",CONFIRM:"Potvrdi"},id:{OK:"OK",CANCEL:"Batal",CONFIRM:"OK"},it:{OK:"OK",CANCEL:"Annulla",CONFIRM:"Conferma"},ja:{OK:"OK",CANCEL:"キャンセル",CONFIRM:"確認"},lt:{OK:"Gerai",CANCEL:"Atšaukti",CONFIRM:"Patvirtinti"},lv:{OK:"Labi",CANCEL:"Atcelt",CONFIRM:"Apstiprināt"},nl:{OK:"OK",CANCEL:"Annuleren",CONFIRM:"Accepteren"},no:{OK:"OK",CANCEL:"Avbryt",CONFIRM:"OK"},pl:{OK:"OK",CANCEL:"Anuluj",CONFIRM:"Potwierdź"},pt:{OK:"OK",CANCEL:"Cancelar",CONFIRM:"Confirmar"},ru:{OK:"OK",CANCEL:"Отмена",CONFIRM:"Применить"},sq:{OK:"OK",CANCEL:"Anulo",CONFIRM:"Prano"},sv:{OK:"OK",CANCEL:"Avbryt",CONFIRM:"OK"},th:{OK:"ตกลง",CANCEL:"ยกเลิก",CONFIRM:"ยืนยัน"},tr:{OK:"Tamam",CANCEL:"İptal",CONFIRM:"Onayla"},zh_CN:{OK:"OK",CANCEL:"取消",CONFIRM:"确认"},zh_TW:{OK:"OK",CANCEL:"取消",CONFIRM:"確認"}};return p.addLocale=function(a,c){return b.each(["OK","CANCEL","CONFIRM"],function(a,b){if(!c[b])throw new Error("Please supply a translation for '"+b+"'")}),q[a]={OK:c.OK,CANCEL:c.CANCEL,CONFIRM:c.CONFIRM},p},p.removeLocale=function(a){return delete q[a],p},p.setLocale=function(a){return p.setDefaults("locale",a)},p.init=function(c){return a(c||b)},p}); -------------------------------------------------------------------------------- /static/js/popper.min.js: -------------------------------------------------------------------------------- 1 | /* 2 | Copyright (C) Federico Zivolo 2018 3 | Distributed under the MIT License (license terms are at http://opensource.org/licenses/MIT). 4 | */(function(e,t){'object'==typeof exports&&'undefined'!=typeof module?module.exports=t():'function'==typeof define&&define.amd?define(t):e.Popper=t()})(this,function(){'use strict';function e(e){return e&&'[object Function]'==={}.toString.call(e)}function t(e,t){if(1!==e.nodeType)return[];var o=getComputedStyle(e,null);return t?o[t]:o}function o(e){return'HTML'===e.nodeName?e:e.parentNode||e.host}function n(e){if(!e)return document.body;switch(e.nodeName){case'HTML':case'BODY':return e.ownerDocument.body;case'#document':return e.body;}var i=t(e),r=i.overflow,p=i.overflowX,s=i.overflowY;return /(auto|scroll|overlay)/.test(r+s+p)?e:n(o(e))}function r(e){return 11===e?re:10===e?pe:re||pe}function p(e){if(!e)return document.documentElement;for(var o=r(10)?document.body:null,n=e.offsetParent;n===o&&e.nextElementSibling;)n=(e=e.nextElementSibling).offsetParent;var i=n&&n.nodeName;return i&&'BODY'!==i&&'HTML'!==i?-1!==['TD','TABLE'].indexOf(n.nodeName)&&'static'===t(n,'position')?p(n):n:e?e.ownerDocument.documentElement:document.documentElement}function s(e){var t=e.nodeName;return'BODY'!==t&&('HTML'===t||p(e.firstElementChild)===e)}function d(e){return null===e.parentNode?e:d(e.parentNode)}function a(e,t){if(!e||!e.nodeType||!t||!t.nodeType)return document.documentElement;var o=e.compareDocumentPosition(t)&Node.DOCUMENT_POSITION_FOLLOWING,n=o?e:t,i=o?t:e,r=document.createRange();r.setStart(n,0),r.setEnd(i,0);var l=r.commonAncestorContainer;if(e!==l&&t!==l||n.contains(i))return s(l)?l:p(l);var f=d(e);return f.host?a(f.host,t):a(e,d(t).host)}function l(e){var t=1<arguments.length&&void 0!==arguments[1]?arguments[1]:'top',o='top'===t?'scrollTop':'scrollLeft',n=e.nodeName;if('BODY'===n||'HTML'===n){var i=e.ownerDocument.documentElement,r=e.ownerDocument.scrollingElement||i;return r[o]}return e[o]}function f(e,t){var o=2<arguments.length&&void 0!==arguments[2]&&arguments[2],n=l(t,'top'),i=l(t,'left'),r=o?-1:1;return e.top+=n*r,e.bottom+=n*r,e.left+=i*r,e.right+=i*r,e}function m(e,t){var o='x'===t?'Left':'Top',n='Left'==o?'Right':'Bottom';return parseFloat(e['border'+o+'Width'],10)+parseFloat(e['border'+n+'Width'],10)}function h(e,t,o,n){return $(t['offset'+e],t['scroll'+e],o['client'+e],o['offset'+e],o['scroll'+e],r(10)?o['offset'+e]+n['margin'+('Height'===e?'Top':'Left')]+n['margin'+('Height'===e?'Bottom':'Right')]:0)}function c(){var e=document.body,t=document.documentElement,o=r(10)&&getComputedStyle(t);return{height:h('Height',e,t,o),width:h('Width',e,t,o)}}function g(e){return le({},e,{right:e.left+e.width,bottom:e.top+e.height})}function u(e){var o={};try{if(r(10)){o=e.getBoundingClientRect();var n=l(e,'top'),i=l(e,'left');o.top+=n,o.left+=i,o.bottom+=n,o.right+=i}else o=e.getBoundingClientRect()}catch(t){}var p={left:o.left,top:o.top,width:o.right-o.left,height:o.bottom-o.top},s='HTML'===e.nodeName?c():{},d=s.width||e.clientWidth||p.right-p.left,a=s.height||e.clientHeight||p.bottom-p.top,f=e.offsetWidth-d,h=e.offsetHeight-a;if(f||h){var u=t(e);f-=m(u,'x'),h-=m(u,'y'),p.width-=f,p.height-=h}return g(p)}function b(e,o){var i=2<arguments.length&&void 0!==arguments[2]&&arguments[2],p=r(10),s='HTML'===o.nodeName,d=u(e),a=u(o),l=n(e),m=t(o),h=parseFloat(m.borderTopWidth,10),c=parseFloat(m.borderLeftWidth,10);i&&'HTML'===o.nodeName&&(a.top=$(a.top,0),a.left=$(a.left,0));var b=g({top:d.top-a.top-h,left:d.left-a.left-c,width:d.width,height:d.height});if(b.marginTop=0,b.marginLeft=0,!p&&s){var y=parseFloat(m.marginTop,10),w=parseFloat(m.marginLeft,10);b.top-=h-y,b.bottom-=h-y,b.left-=c-w,b.right-=c-w,b.marginTop=y,b.marginLeft=w}return(p&&!i?o.contains(l):o===l&&'BODY'!==l.nodeName)&&(b=f(b,o)),b}function y(e){var t=1<arguments.length&&void 0!==arguments[1]&&arguments[1],o=e.ownerDocument.documentElement,n=b(e,o),i=$(o.clientWidth,window.innerWidth||0),r=$(o.clientHeight,window.innerHeight||0),p=t?0:l(o),s=t?0:l(o,'left'),d={top:p-n.top+n.marginTop,left:s-n.left+n.marginLeft,width:i,height:r};return g(d)}function w(e){var n=e.nodeName;return'BODY'===n||'HTML'===n?!1:'fixed'===t(e,'position')||w(o(e))}function E(e){if(!e||!e.parentElement||r())return document.documentElement;for(var o=e.parentElement;o&&'none'===t(o,'transform');)o=o.parentElement;return o||document.documentElement}function v(e,t,i,r){var p=4<arguments.length&&void 0!==arguments[4]&&arguments[4],s={top:0,left:0},d=p?E(e):a(e,t);if('viewport'===r)s=y(d,p);else{var l;'scrollParent'===r?(l=n(o(t)),'BODY'===l.nodeName&&(l=e.ownerDocument.documentElement)):'window'===r?l=e.ownerDocument.documentElement:l=r;var f=b(l,d,p);if('HTML'===l.nodeName&&!w(d)){var m=c(),h=m.height,g=m.width;s.top+=f.top-f.marginTop,s.bottom=h+f.top,s.left+=f.left-f.marginLeft,s.right=g+f.left}else s=f}return s.left+=i,s.top+=i,s.right-=i,s.bottom-=i,s}function x(e){var t=e.width,o=e.height;return t*o}function O(e,t,o,n,i){var r=5<arguments.length&&void 0!==arguments[5]?arguments[5]:0;if(-1===e.indexOf('auto'))return e;var p=v(o,n,r,i),s={top:{width:p.width,height:t.top-p.top},right:{width:p.right-t.right,height:p.height},bottom:{width:p.width,height:p.bottom-t.bottom},left:{width:t.left-p.left,height:p.height}},d=Object.keys(s).map(function(e){return le({key:e},s[e],{area:x(s[e])})}).sort(function(e,t){return t.area-e.area}),a=d.filter(function(e){var t=e.width,n=e.height;return t>=o.clientWidth&&n>=o.clientHeight}),l=0<a.length?a[0].key:d[0].key,f=e.split('-')[1];return l+(f?'-'+f:'')}function L(e,t,o){var n=3<arguments.length&&void 0!==arguments[3]?arguments[3]:null,i=n?E(t):a(t,o);return b(o,i,n)}function S(e){var t=getComputedStyle(e),o=parseFloat(t.marginTop)+parseFloat(t.marginBottom),n=parseFloat(t.marginLeft)+parseFloat(t.marginRight),i={width:e.offsetWidth+n,height:e.offsetHeight+o};return i}function T(e){var t={left:'right',right:'left',bottom:'top',top:'bottom'};return e.replace(/left|right|bottom|top/g,function(e){return t[e]})}function C(e,t,o){o=o.split('-')[0];var n=S(e),i={width:n.width,height:n.height},r=-1!==['right','left'].indexOf(o),p=r?'top':'left',s=r?'left':'top',d=r?'height':'width',a=r?'width':'height';return i[p]=t[p]+t[d]/2-n[d]/2,i[s]=o===s?t[s]-n[a]:t[T(s)],i}function D(e,t){return Array.prototype.find?e.find(t):e.filter(t)[0]}function N(e,t,o){if(Array.prototype.findIndex)return e.findIndex(function(e){return e[t]===o});var n=D(e,function(e){return e[t]===o});return e.indexOf(n)}function P(t,o,n){var i=void 0===n?t:t.slice(0,N(t,'name',n));return i.forEach(function(t){t['function']&&console.warn('`modifier.function` is deprecated, use `modifier.fn`!');var n=t['function']||t.fn;t.enabled&&e(n)&&(o.offsets.popper=g(o.offsets.popper),o.offsets.reference=g(o.offsets.reference),o=n(o,t))}),o}function k(){if(!this.state.isDestroyed){var e={instance:this,styles:{},arrowStyles:{},attributes:{},flipped:!1,offsets:{}};e.offsets.reference=L(this.state,this.popper,this.reference,this.options.positionFixed),e.placement=O(this.options.placement,e.offsets.reference,this.popper,this.reference,this.options.modifiers.flip.boundariesElement,this.options.modifiers.flip.padding),e.originalPlacement=e.placement,e.positionFixed=this.options.positionFixed,e.offsets.popper=C(this.popper,e.offsets.reference,e.placement),e.offsets.popper.position=this.options.positionFixed?'fixed':'absolute',e=P(this.modifiers,e),this.state.isCreated?this.options.onUpdate(e):(this.state.isCreated=!0,this.options.onCreate(e))}}function W(e,t){return e.some(function(e){var o=e.name,n=e.enabled;return n&&o===t})}function B(e){for(var t=[!1,'ms','Webkit','Moz','O'],o=e.charAt(0).toUpperCase()+e.slice(1),n=0;n<t.length;n++){var i=t[n],r=i?''+i+o:e;if('undefined'!=typeof document.body.style[r])return r}return null}function H(){return this.state.isDestroyed=!0,W(this.modifiers,'applyStyle')&&(this.popper.removeAttribute('x-placement'),this.popper.style.position='',this.popper.style.top='',this.popper.style.left='',this.popper.style.right='',this.popper.style.bottom='',this.popper.style.willChange='',this.popper.style[B('transform')]=''),this.disableEventListeners(),this.options.removeOnDestroy&&this.popper.parentNode.removeChild(this.popper),this}function A(e){var t=e.ownerDocument;return t?t.defaultView:window}function M(e,t,o,i){var r='BODY'===e.nodeName,p=r?e.ownerDocument.defaultView:e;p.addEventListener(t,o,{passive:!0}),r||M(n(p.parentNode),t,o,i),i.push(p)}function I(e,t,o,i){o.updateBound=i,A(e).addEventListener('resize',o.updateBound,{passive:!0});var r=n(e);return M(r,'scroll',o.updateBound,o.scrollParents),o.scrollElement=r,o.eventsEnabled=!0,o}function F(){this.state.eventsEnabled||(this.state=I(this.reference,this.options,this.state,this.scheduleUpdate))}function R(e,t){return A(e).removeEventListener('resize',t.updateBound),t.scrollParents.forEach(function(e){e.removeEventListener('scroll',t.updateBound)}),t.updateBound=null,t.scrollParents=[],t.scrollElement=null,t.eventsEnabled=!1,t}function U(){this.state.eventsEnabled&&(cancelAnimationFrame(this.scheduleUpdate),this.state=R(this.reference,this.state))}function Y(e){return''!==e&&!isNaN(parseFloat(e))&&isFinite(e)}function j(e,t){Object.keys(t).forEach(function(o){var n='';-1!==['width','height','top','right','bottom','left'].indexOf(o)&&Y(t[o])&&(n='px'),e.style[o]=t[o]+n})}function K(e,t){Object.keys(t).forEach(function(o){var n=t[o];!1===n?e.removeAttribute(o):e.setAttribute(o,t[o])})}function q(e,t,o){var n=D(e,function(e){var o=e.name;return o===t}),i=!!n&&e.some(function(e){return e.name===o&&e.enabled&&e.order<n.order});if(!i){var r='`'+t+'`';console.warn('`'+o+'`'+' modifier is required by '+r+' modifier in order to work, be sure to include it before '+r+'!')}return i}function G(e){return'end'===e?'start':'start'===e?'end':e}function z(e){var t=1<arguments.length&&void 0!==arguments[1]&&arguments[1],o=me.indexOf(e),n=me.slice(o+1).concat(me.slice(0,o));return t?n.reverse():n}function V(e,t,o,n){var i=e.match(/((?:\-|\+)?\d*\.?\d*)(.*)/),r=+i[1],p=i[2];if(!r)return e;if(0===p.indexOf('%')){var s;switch(p){case'%p':s=o;break;case'%':case'%r':default:s=n;}var d=g(s);return d[t]/100*r}if('vh'===p||'vw'===p){var a;return a='vh'===p?$(document.documentElement.clientHeight,window.innerHeight||0):$(document.documentElement.clientWidth,window.innerWidth||0),a/100*r}return r}function _(e,t,o,n){var i=[0,0],r=-1!==['right','left'].indexOf(n),p=e.split(/(\+|\-)/).map(function(e){return e.trim()}),s=p.indexOf(D(p,function(e){return-1!==e.search(/,|\s/)}));p[s]&&-1===p[s].indexOf(',')&&console.warn('Offsets separated by white space(s) are deprecated, use a comma (,) instead.');var d=/\s*,\s*|\s+/,a=-1===s?[p]:[p.slice(0,s).concat([p[s].split(d)[0]]),[p[s].split(d)[1]].concat(p.slice(s+1))];return a=a.map(function(e,n){var i=(1===n?!r:r)?'height':'width',p=!1;return e.reduce(function(e,t){return''===e[e.length-1]&&-1!==['+','-'].indexOf(t)?(e[e.length-1]=t,p=!0,e):p?(e[e.length-1]+=t,p=!1,e):e.concat(t)},[]).map(function(e){return V(e,i,t,o)})}),a.forEach(function(e,t){e.forEach(function(o,n){Y(o)&&(i[t]+=o*('-'===e[n-1]?-1:1))})}),i}function X(e,t){var o,n=t.offset,i=e.placement,r=e.offsets,p=r.popper,s=r.reference,d=i.split('-')[0];return o=Y(+n)?[+n,0]:_(n,p,s,d),'left'===d?(p.top+=o[0],p.left-=o[1]):'right'===d?(p.top+=o[0],p.left+=o[1]):'top'===d?(p.left+=o[0],p.top-=o[1]):'bottom'===d&&(p.left+=o[0],p.top+=o[1]),e.popper=p,e}for(var J=Math.min,Q=Math.round,Z=Math.floor,$=Math.max,ee='undefined'!=typeof window&&'undefined'!=typeof document,te=['Edge','Trident','Firefox'],oe=0,ne=0;ne<te.length;ne+=1)if(ee&&0<=navigator.userAgent.indexOf(te[ne])){oe=1;break}var i=ee&&window.Promise,ie=i?function(e){var t=!1;return function(){t||(t=!0,window.Promise.resolve().then(function(){t=!1,e()}))}}:function(e){var t=!1;return function(){t||(t=!0,setTimeout(function(){t=!1,e()},oe))}},re=ee&&!!(window.MSInputMethodContext&&document.documentMode),pe=ee&&/MSIE 10/.test(navigator.userAgent),se=function(e,t){if(!(e instanceof t))throw new TypeError('Cannot call a class as a function')},de=function(){function e(e,t){for(var o,n=0;n<t.length;n++)o=t[n],o.enumerable=o.enumerable||!1,o.configurable=!0,'value'in o&&(o.writable=!0),Object.defineProperty(e,o.key,o)}return function(t,o,n){return o&&e(t.prototype,o),n&&e(t,n),t}}(),ae=function(e,t,o){return t in e?Object.defineProperty(e,t,{value:o,enumerable:!0,configurable:!0,writable:!0}):e[t]=o,e},le=Object.assign||function(e){for(var t,o=1;o<arguments.length;o++)for(var n in t=arguments[o],t)Object.prototype.hasOwnProperty.call(t,n)&&(e[n]=t[n]);return e},fe=['auto-start','auto','auto-end','top-start','top','top-end','right-start','right','right-end','bottom-end','bottom','bottom-start','left-end','left','left-start'],me=fe.slice(3),he={FLIP:'flip',CLOCKWISE:'clockwise',COUNTERCLOCKWISE:'counterclockwise'},ce=function(){function t(o,n){var i=this,r=2<arguments.length&&void 0!==arguments[2]?arguments[2]:{};se(this,t),this.scheduleUpdate=function(){return requestAnimationFrame(i.update)},this.update=ie(this.update.bind(this)),this.options=le({},t.Defaults,r),this.state={isDestroyed:!1,isCreated:!1,scrollParents:[]},this.reference=o&&o.jquery?o[0]:o,this.popper=n&&n.jquery?n[0]:n,this.options.modifiers={},Object.keys(le({},t.Defaults.modifiers,r.modifiers)).forEach(function(e){i.options.modifiers[e]=le({},t.Defaults.modifiers[e]||{},r.modifiers?r.modifiers[e]:{})}),this.modifiers=Object.keys(this.options.modifiers).map(function(e){return le({name:e},i.options.modifiers[e])}).sort(function(e,t){return e.order-t.order}),this.modifiers.forEach(function(t){t.enabled&&e(t.onLoad)&&t.onLoad(i.reference,i.popper,i.options,t,i.state)}),this.update();var p=this.options.eventsEnabled;p&&this.enableEventListeners(),this.state.eventsEnabled=p}return de(t,[{key:'update',value:function(){return k.call(this)}},{key:'destroy',value:function(){return H.call(this)}},{key:'enableEventListeners',value:function(){return F.call(this)}},{key:'disableEventListeners',value:function(){return U.call(this)}}]),t}();return ce.Utils=('undefined'==typeof window?global:window).PopperUtils,ce.placements=fe,ce.Defaults={placement:'bottom',positionFixed:!1,eventsEnabled:!0,removeOnDestroy:!1,onCreate:function(){},onUpdate:function(){},modifiers:{shift:{order:100,enabled:!0,fn:function(e){var t=e.placement,o=t.split('-')[0],n=t.split('-')[1];if(n){var i=e.offsets,r=i.reference,p=i.popper,s=-1!==['bottom','top'].indexOf(o),d=s?'left':'top',a=s?'width':'height',l={start:ae({},d,r[d]),end:ae({},d,r[d]+r[a]-p[a])};e.offsets.popper=le({},p,l[n])}return e}},offset:{order:200,enabled:!0,fn:X,offset:0},preventOverflow:{order:300,enabled:!0,fn:function(e,t){var o=t.boundariesElement||p(e.instance.popper);e.instance.reference===o&&(o=p(o));var n=B('transform'),i=e.instance.popper.style,r=i.top,s=i.left,d=i[n];i.top='',i.left='',i[n]='';var a=v(e.instance.popper,e.instance.reference,t.padding,o,e.positionFixed);i.top=r,i.left=s,i[n]=d,t.boundaries=a;var l=t.priority,f=e.offsets.popper,m={primary:function(e){var o=f[e];return f[e]<a[e]&&!t.escapeWithReference&&(o=$(f[e],a[e])),ae({},e,o)},secondary:function(e){var o='right'===e?'left':'top',n=f[o];return f[e]>a[e]&&!t.escapeWithReference&&(n=J(f[o],a[e]-('right'===e?f.width:f.height))),ae({},o,n)}};return l.forEach(function(e){var t=-1===['left','top'].indexOf(e)?'secondary':'primary';f=le({},f,m[t](e))}),e.offsets.popper=f,e},priority:['left','right','top','bottom'],padding:5,boundariesElement:'scrollParent'},keepTogether:{order:400,enabled:!0,fn:function(e){var t=e.offsets,o=t.popper,n=t.reference,i=e.placement.split('-')[0],r=Z,p=-1!==['top','bottom'].indexOf(i),s=p?'right':'bottom',d=p?'left':'top',a=p?'width':'height';return o[s]<r(n[d])&&(e.offsets.popper[d]=r(n[d])-o[a]),o[d]>r(n[s])&&(e.offsets.popper[d]=r(n[s])),e}},arrow:{order:500,enabled:!0,fn:function(e,o){var n;if(!q(e.instance.modifiers,'arrow','keepTogether'))return e;var i=o.element;if('string'==typeof i){if(i=e.instance.popper.querySelector(i),!i)return e;}else if(!e.instance.popper.contains(i))return console.warn('WARNING: `arrow.element` must be child of its popper element!'),e;var r=e.placement.split('-')[0],p=e.offsets,s=p.popper,d=p.reference,a=-1!==['left','right'].indexOf(r),l=a?'height':'width',f=a?'Top':'Left',m=f.toLowerCase(),h=a?'left':'top',c=a?'bottom':'right',u=S(i)[l];d[c]-u<s[m]&&(e.offsets.popper[m]-=s[m]-(d[c]-u)),d[m]+u>s[c]&&(e.offsets.popper[m]+=d[m]+u-s[c]),e.offsets.popper=g(e.offsets.popper);var b=d[m]+d[l]/2-u/2,y=t(e.instance.popper),w=parseFloat(y['margin'+f],10),E=parseFloat(y['border'+f+'Width'],10),v=b-e.offsets.popper[m]-w-E;return v=$(J(s[l]-u,v),0),e.arrowElement=i,e.offsets.arrow=(n={},ae(n,m,Q(v)),ae(n,h,''),n),e},element:'[x-arrow]'},flip:{order:600,enabled:!0,fn:function(e,t){if(W(e.instance.modifiers,'inner'))return e;if(e.flipped&&e.placement===e.originalPlacement)return e;var o=v(e.instance.popper,e.instance.reference,t.padding,t.boundariesElement,e.positionFixed),n=e.placement.split('-')[0],i=T(n),r=e.placement.split('-')[1]||'',p=[];switch(t.behavior){case he.FLIP:p=[n,i];break;case he.CLOCKWISE:p=z(n);break;case he.COUNTERCLOCKWISE:p=z(n,!0);break;default:p=t.behavior;}return p.forEach(function(s,d){if(n!==s||p.length===d+1)return e;n=e.placement.split('-')[0],i=T(n);var a=e.offsets.popper,l=e.offsets.reference,f=Z,m='left'===n&&f(a.right)>f(l.left)||'right'===n&&f(a.left)<f(l.right)||'top'===n&&f(a.bottom)>f(l.top)||'bottom'===n&&f(a.top)<f(l.bottom),h=f(a.left)<f(o.left),c=f(a.right)>f(o.right),g=f(a.top)<f(o.top),u=f(a.bottom)>f(o.bottom),b='left'===n&&h||'right'===n&&c||'top'===n&&g||'bottom'===n&&u,y=-1!==['top','bottom'].indexOf(n),w=!!t.flipVariations&&(y&&'start'===r&&h||y&&'end'===r&&c||!y&&'start'===r&&g||!y&&'end'===r&&u);(m||b||w)&&(e.flipped=!0,(m||b)&&(n=p[d+1]),w&&(r=G(r)),e.placement=n+(r?'-'+r:''),e.offsets.popper=le({},e.offsets.popper,C(e.instance.popper,e.offsets.reference,e.placement)),e=P(e.instance.modifiers,e,'flip'))}),e},behavior:'flip',padding:5,boundariesElement:'viewport'},inner:{order:700,enabled:!1,fn:function(e){var t=e.placement,o=t.split('-')[0],n=e.offsets,i=n.popper,r=n.reference,p=-1!==['left','right'].indexOf(o),s=-1===['top','left'].indexOf(o);return i[p?'left':'top']=r[o]-(s?i[p?'width':'height']:0),e.placement=T(t),e.offsets.popper=g(i),e}},hide:{order:800,enabled:!0,fn:function(e){if(!q(e.instance.modifiers,'hide','preventOverflow'))return e;var t=e.offsets.reference,o=D(e.instance.modifiers,function(e){return'preventOverflow'===e.name}).boundaries;if(t.bottom<o.top||t.left>o.right||t.top>o.bottom||t.right<o.left){if(!0===e.hide)return e;e.hide=!0,e.attributes['x-out-of-boundaries']=''}else{if(!1===e.hide)return e;e.hide=!1,e.attributes['x-out-of-boundaries']=!1}return e}},computeStyle:{order:850,enabled:!0,fn:function(e,t){var o=t.x,n=t.y,i=e.offsets.popper,r=D(e.instance.modifiers,function(e){return'applyStyle'===e.name}).gpuAcceleration;void 0!==r&&console.warn('WARNING: `gpuAcceleration` option moved to `computeStyle` modifier and will not be supported in future versions of Popper.js!');var s,d,a=void 0===r?t.gpuAcceleration:r,l=p(e.instance.popper),f=u(l),m={position:i.position},h={left:Z(i.left),top:Q(i.top),bottom:Q(i.bottom),right:Z(i.right)},c='bottom'===o?'top':'bottom',g='right'===n?'left':'right',b=B('transform');if(d='bottom'==c?-f.height+h.bottom:h.top,s='right'==g?-f.width+h.right:h.left,a&&b)m[b]='translate3d('+s+'px, '+d+'px, 0)',m[c]=0,m[g]=0,m.willChange='transform';else{var y='bottom'==c?-1:1,w='right'==g?-1:1;m[c]=d*y,m[g]=s*w,m.willChange=c+', '+g}var E={"x-placement":e.placement};return e.attributes=le({},E,e.attributes),e.styles=le({},m,e.styles),e.arrowStyles=le({},e.offsets.arrow,e.arrowStyles),e},gpuAcceleration:!0,x:'bottom',y:'right'},applyStyle:{order:900,enabled:!0,fn:function(e){return j(e.instance.popper,e.styles),K(e.instance.popper,e.attributes),e.arrowElement&&Object.keys(e.arrowStyles).length&&j(e.arrowElement,e.arrowStyles),e},onLoad:function(e,t,o,n,i){var r=L(i,t,e,o.positionFixed),p=O(o.placement,r,t,e,o.modifiers.flip.boundariesElement,o.modifiers.flip.padding);return t.setAttribute('x-placement',p),j(t,{position:o.positionFixed?'fixed':'absolute'}),o},gpuAcceleration:void 0}}},ce}); 5 | //# sourceMappingURL=popper.min.js.map 6 | -------------------------------------------------------------------------------- /static/menu.svg: -------------------------------------------------------------------------------- 1 | <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-menu"><line x1="3" y1="12" x2="21" y2="12"></line><line x1="3" y1="6" x2="21" y2="6"></line><line x1="3" y1="18" x2="21" y2="18"></line></svg> -------------------------------------------------------------------------------- /templates/archive_not_found.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% load static %} 4 | 5 | {% block stylesheet %} 6 | <link rel="stylesheet" href="{% static 'css/accounts.css' %}"> 7 | {% endblock %} 8 | 9 | {% block body %} 10 | <div class="container"> 11 | <h1 class="text-center logo my-4"> 12 | <a href="{{path}}">Back</a> 13 | </h1> 14 | <div class="row justify-content-center"> 15 | <div class="col-lg-5 col-md-10 col-sm-12"> 16 | <div class="alert alert-warning text-center" role="alert"> 17 | Sorry! File has not been archived in this format. 18 | Consider archiving it again. If there is any problem 19 | in default archiving method, then try using 20 | Chromium backend for generating HTML/PDF. 21 | </div> 22 | </div> 23 | </div> 24 | </div> 25 | {% endblock %} 26 | -------------------------------------------------------------------------------- /templates/base.html: -------------------------------------------------------------------------------- 1 | {% load static %}<!DOCTYPE html> 2 | <html> 3 | <head> 4 | <meta charset="utf-8"> 5 | <title>{% block title %}Reminiscence{% endblock %} 6 | 7 | 8 | 9 | 10 | {% block body %} 11 | 12 | 49 |
50 |
    51 | {% block breadcrumb %} 52 | {% endblock %} 53 |
54 | {% block content %} 55 | {% endblock %} 56 |
57 | {% endblock body %} 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | -------------------------------------------------------------------------------- /templates/home.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% load widget_tweaks %} 4 | 5 | {% load static %} 6 | 7 | {% block breadcrumb %} 8 |
  • Home
    • 9 | {% endblock %} 10 | 11 | 12 | {% block content %} 13 | 14 |
    15 | {% csrf_token %} 16 | 17 | {% for hidden in form.hidden_fields %} 18 | {{ hidden }} 19 | {% endfor %} 20 | 21 | {% for field in form.visible_fields %} 22 |
    23 |
    24 | {{ field| add_class:'form-control'}} 25 | {% for error in field.errors %} 26 | {{ error }} 27 | {% endfor %} 28 |
    29 | 30 |
    31 | {% endfor %} 32 |
    33 |
    34 |
    35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | {% for index, key, value, loc, rename_link, remove_link in usr_list %} 46 | 47 | 48 | 51 | 54 | 55 | 67 | 68 | 69 | {% endfor %} 70 | 71 |
    DirectoryLinksAction
    49 | {{ key }} 50 | 52 | {{ value }} 53 | 56 | 57 |
    58 | 59 | 64 |
    65 | 66 |
    72 |
    73 | {% endblock %} 74 | 75 | -------------------------------------------------------------------------------- /templates/home_dir.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% load widget_tweaks %} 4 | 5 | {% load static %} 6 | 7 | {% block breadcrumb %} 8 | {% for is_active, dir_name, path in dir_list %} 9 |
  • 10 | {{dir_name}} 11 |
    • 12 | {% endfor %} 13 | {% endblock %} 14 | 15 | 16 | {% block content %} 17 | 18 |
    19 | {% csrf_token %} 20 | 21 | {% for hidden in form.hidden_fields %} 22 | {{ hidden }} 23 | {% endfor %} 24 | 25 | {% for field in form.visible_fields %} 26 |
    27 |
    28 | {{ field| add_class:'form-control'}} 29 | {% for error in field.errors %} 30 | {{ error }} 31 | {% endfor %} 32 |
    33 | 34 |
    35 | {% endfor %} 36 |
    37 |
    38 |
    39 | 40 | 41 | 42 | 44 | 45 | 65 | 66 | 67 | 68 | {% for index, title, netloc, loc, edit_b, remove_link, timestamp, taglist, ms, mm , archive, dir, rurl, idd, fav_path, media_element, is_subdir, rename_link in usr_list %} 69 | 70 | 71 | 86 | 122 | 123 | {% endfor %} 124 | 125 |
    43 | Title 46 |
    47 | 49 |
    50 | Create Sub-Dir 51 |
    52 | Select Multiple 53 | Select All 54 |
    55 | Move Selected 56 | Archive Selected 57 | Edit Tags 58 |
    59 | Merge With 60 |
    61 | Media Playlist 62 |
    63 |
    64 |
    {% if fav_path %}{% endif %} 72 | {{title}}
    73 | {% if not is_subdir %} 74 | 75 | {{netloc}} 76 | 77 | {% if taglist %} 78 |
    79 | {% for tag in taglist %} 80 | {{tag}} 81 | {% endfor %} 82 |
    83 | {% endif %} 84 | {% endif %} 85 |     		87 |
    88 | 89 | 114 |
    115 |
    116 | 117 | {% if not is_subdir %} 118 |
    {{timestamp}}
    119 | {% endif %} 120 |     121 |
    126 |
    127 | 152 | {% endblock %} 153 | 154 | 155 | 156 | -------------------------------------------------------------------------------- /templates/includes/form.html: -------------------------------------------------------------------------------- 1 | {% load widget_tweaks %} 2 | 3 | {% for field in form %} 4 |
    5 | 6 | {{ field.label_tag }} 7 | 8 | {% if form.is_bound %} 9 | {% if field.errors %} 10 | {% render_field field class="form-control is-invalid" %} 11 | {% for error in field.errors %} 12 |
    13 | {{ error }} 14 |
    15 | {% endfor %} 16 | {% else %} 17 | {% render_field field class="form-control is-valid" %} 18 | {% endif %} 19 | {% else %} 20 | {% render_field field class="form-control" %} 21 | {% endif %} 22 | 23 | {% if field.help_text %} 24 | 25 | {{ field.help_text| safe}} 26 | 27 | {% endif %} 28 |
    29 | {% endfor %} 30 | -------------------------------------------------------------------------------- /templates/login.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% load static %} 4 | 5 | {% block stylesheet %} 6 | 7 | {% endblock %} 8 | 9 | {% block body %} 10 |
    11 |

    12 | Reminiscence 13 |

    14 |
    15 |
    16 |
    17 |
    18 |
    19 | {% csrf_token %} 20 | 21 | {% include 'includes/form.html' %} 22 | 23 |
    24 |
    25 |
    26 | New to Reminiscence? Sign up 27 |
    28 |
    29 |
    30 | 31 | Forgot your password? 32 | 33 |
    34 |
    35 |
    36 |
    37 | {% endblock %} 38 | -------------------------------------------------------------------------------- /templates/password_change.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% block title %}Change password{% endblock %} 4 | 5 | {% block breadcrumb %} 6 |
  • Change password
    • 7 | {% endblock %} 8 | 9 | {% block content %} 10 | 11 |
    12 |
    13 |
    14 | {% csrf_token %} 15 | {% include 'includes/form.html' %} 16 | 17 |
    18 |
    19 |
    20 | {% endblock %} 21 | -------------------------------------------------------------------------------- /templates/password_change_done.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% block title %}Change password successful{% endblock %} 4 | 5 | {% block breadcrumb %} 6 |
  • Change password
    • 7 |
  • Success
    • 8 | {% endblock %} 9 | 10 | {% block content %} 11 |
    12 |
    13 | 16 | Return to home page 17 |
    18 |
    19 | {% endblock %} 20 | -------------------------------------------------------------------------------- /templates/public.html: -------------------------------------------------------------------------------- 1 | {% load static %} 2 | 3 | 4 | 5 | 6 | 7 | {% block title %}Reminiscence{% endblock %} 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | {% block content %} 16 | 17 |
    18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | {% for index, title, netloc, loc, edit_b, remove_link, timestamp, taglist, ms, mm , archive, dir, rurl, idd, fav_path, media_element, is_subdir, rename_link in usr_list %} 28 | 29 | 30 | 45 | 54 | 55 | {% endfor %} 56 | 57 |
    TitleAction
    {% if fav_path %}{% endif %} 31 | {{title}}
    32 | {% if not is_subdir %} 33 | 34 | {{netloc}} 35 | 36 | {% if taglist %} 37 |
    38 | {% for tag in taglist %} 39 | {{tag}} 40 | {% endfor %} 41 |
    42 | {% endif %} 43 | {% endif %} 44 |     		46 |
    47 | 48 |
    49 |
    50 | 51 |
    {{timestamp}}
    52 |     53 |
    58 |
    59 | {% endblock %} 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kanishka-linux/reminiscence/bd52d1641a819b5054d384be7a4a9cba42e16781/tests/__init__.py -------------------------------------------------------------------------------- /tests/tests_drf.py: -------------------------------------------------------------------------------- 1 | from django.contrib.auth.models import User 2 | from django.test import TestCase, Client, tag 3 | from django.urls import resolve, reverse 4 | from pages.models import Library 5 | from django.utils import timezone 6 | 7 | class DRFTests(TestCase): 8 | 9 | client = Client() 10 | iurl = 'https://en.wikipedia.org/wiki/Main_Page' 11 | auth_token = None 12 | 13 | @classmethod 14 | def setUpTestData(cls): 15 | usr = User.objects.create_user(username='johndoe', password='clrsalgo') 16 | Library.objects.create(usr=usr, directory='TMP') 17 | Library.objects.create(usr=usr, directory='TMP', title='Wiki', 18 | url=cls.iurl, timestamp=timezone.now()) 19 | 20 | def setUp(self): 21 | url = reverse('get_auth_token') 22 | response = self.client.post(url, {'username': 'johndoe', 'password': 'clrsalgo'}) 23 | self.auth_token = response.json().get('token') 24 | 25 | @tag('async') 26 | def test_add_url(self): 27 | url = reverse('add_url') 28 | post_data = { 29 | "url": "https://mr.wikipedia.org/wiki/Main_Page", 30 | "media_link": "no", "directory": "/TMP", 31 | "save_favicon": "no" 32 | } 33 | response = self.client.post(url, post_data, HTTP_AUTHORIZATION='Token {}'.format(self.auth_token)) 34 | self.assertEquals(response.status_code, 200) 35 | 36 | def test_drf_list_directories(self): 37 | url = reverse('list_directories') 38 | response = self.client.get(url, HTTP_AUTHORIZATION='Token {}'.format(self.auth_token)) 39 | self.assertEquals(response.status_code, 200) 40 | 41 | def test_drf_list_urls(self): 42 | url = reverse('list_added_urls') 43 | post_data = {"directory": "/TMP"} 44 | response = self.client.post(url, post_data, HTTP_AUTHORIZATION='Token {}'.format(self.auth_token)) 45 | self.assertEquals(response.status_code, 200) 46 | 47 | -------------------------------------------------------------------------------- /tests/tests_home.py: -------------------------------------------------------------------------------- 1 | from django.urls import resolve, reverse 2 | from django.test import TestCase 3 | from pages.views import dashboard 4 | from django.conf import settings 5 | 6 | class HomeTests(TestCase): 7 | 8 | def test_home_view_status_code(self): 9 | url = reverse('home') 10 | response = self.client.get(url) 11 | self.assertEquals(response.status_code, 302) 12 | 13 | def test_home_url_resolves_home_view(self): 14 | view = resolve('{}/'.format(settings.ROOT_URL_LOCATION)) 15 | self.assertEquals(view.func, dashboard) 16 | 17 | -------------------------------------------------------------------------------- /tests/tests_signup.py: -------------------------------------------------------------------------------- 1 | from django.contrib.auth.forms import UserCreationForm 2 | from django.urls import resolve, reverse 3 | from django.test import TestCase 4 | from accounts.views import signup 5 | from django.conf import settings 6 | 7 | 8 | class SignUpTests(TestCase): 9 | 10 | def setUp(self): 11 | url = reverse('signup') 12 | self.response = self.client.get(url) 13 | 14 | def test_signup_status_code(self): 15 | self.assertEquals(self.response.status_code, 200) 16 | 17 | def test_signup_url_resolves_signup_view(self): 18 | view = resolve('{}/signup/'.format(settings.ROOT_URL_LOCATION)) 19 | self.assertEquals(view.func, signup) 20 | 21 | def test_csrf(self): 22 | if settings.ALLOW_ANY_ONE_SIGNUP: 23 | self.assertContains(self.response, 'csrfmiddlewaretoken') 24 | else: 25 | self.assertContains(self.response, 'New sign up not allowed') 26 | 27 | def test_contains_form(self): 28 | if settings.ALLOW_ANY_ONE_SIGNUP: 29 | form = self.response.context.get('form') 30 | self.assertIsInstance(form, UserCreationForm) 31 | else: 32 | self.assertContains(self.response, 'New sign up not allowed') 33 | -------------------------------------------------------------------------------- /tests/tests_sync.py: -------------------------------------------------------------------------------- 1 | from django.contrib.auth.models import User 2 | from django.test import TestCase, Client 3 | from django.urls import resolve, reverse 4 | from pages.models import Library 5 | from pages.views import dashboard 6 | from django.utils import timezone 7 | 8 | 9 | class LibraryTests(TestCase): 10 | 11 | client = Client() 12 | url = 'https://en.wikipedia.org/wiki/Main_Page' 13 | 14 | @classmethod 15 | def setUpTestData(cls): 16 | usr = User.objects.create_user(username='johndoe', password='clrsalgo') 17 | Library.objects.create(usr=usr, directory='TMP') 18 | Library.objects.create(usr=usr, directory='TMP', title='Wiki', 19 | url=cls.url, timestamp=timezone.now()) 20 | 21 | def setUp(self): 22 | self.client.login(username='johndoe', password='clrsalgo') 23 | 24 | def test_dashboard_page(self): 25 | url = reverse('home_page', kwargs={'username': 'johndoe'}) 26 | response = self.client.get(url) 27 | self.assertEquals(response.status_code, 200) 28 | 29 | def test_add_directory(self): 30 | url = reverse('home_page', kwargs={'username': 'johndoe'}) 31 | response = self.client.post(url, {'create_directory':'Sample'}) 32 | self.assertEquals(response.status_code, 200) 33 | 34 | def test_check_url(self): 35 | url = reverse('navigate_directory', kwargs={'username': 'johndoe', 'directory': 'TMP'}) 36 | response = self.client.get(url) 37 | self.assertContains(response, self.url) 38 | 39 | 40 | -------------------------------------------------------------------------------- /vinanti/__init__.py: -------------------------------------------------------------------------------- 1 | from vinanti.vinanti import Vinanti 2 | 3 | __version__ = '0.3' 4 | -------------------------------------------------------------------------------- /vinanti/crawl.py: -------------------------------------------------------------------------------- 1 | import urllib.parse 2 | from urllib.parse import urlparse 3 | from collections import OrderedDict 4 | 5 | try: 6 | from bs4 import BeautifulSoup 7 | except ImportError: 8 | pass 9 | 10 | try: 11 | from vinanti.utils import URL 12 | from vinanti.log import log_function 13 | except ImportError: 14 | from utils import URL 15 | from log import log_function 16 | 17 | logger = log_function(__name__) 18 | 19 | class CrawlObject: 20 | 21 | def __init__(self, vnt, url_obj, onfinished, all_domain, 22 | domains_allowed, depth_allowed): 23 | url = url_obj.url 24 | self.url_obj = url_obj 25 | ourl = urllib.parse.urlparse(url) 26 | self.scheme = ourl.scheme 27 | self.netloc = ourl.netloc 28 | self.vnt = vnt 29 | self.base_url = url 30 | self.ourl = url 31 | if ourl.path and not url.endswith('/'): 32 | self.base_url, _ = self.base_url.rsplit('/', 1) 33 | self.crawl_dict = OrderedDict() 34 | self.onfinished = onfinished 35 | self.link_set = set() 36 | if not self.base_url.endswith('/'): 37 | self.base_url = self.base_url + '/' 38 | if all_domain: 39 | self.all_domain = True 40 | else: 41 | self.all_domain = False 42 | dms = [] 43 | if domains_allowed: 44 | if isinstance(domains_allowed, str): 45 | self.domains_allowed = (self.netloc, domains_allowed) 46 | else: 47 | dms = [i for i in domains_allowed] 48 | self.domains_allowed = (self.netloc, *dms, ) 49 | else: 50 | self.domains_allowed = (self.netloc,) 51 | if isinstance(depth_allowed, int) and depth_allowed > 0: 52 | self.depth_allowed = depth_allowed 53 | else: 54 | self.depth_allowed = 0 55 | 56 | def start_crawling(self, result, url_obj, session): 57 | depth = url_obj.depth 58 | url = url_obj.url 59 | if '#' in url: 60 | pre, ext = url.rsplit('#', 1) 61 | if '/' not in ext and pre: 62 | url = pre 63 | ourl = urllib.parse.urlparse(url) 64 | scheme = ourl.scheme 65 | netloc = ourl.netloc 66 | base_url = url 67 | 68 | if ourl.path and not url.endswith('/'): 69 | base_url, _ = base_url.rsplit('/', 1) 70 | 71 | if not base_url.endswith('/'): 72 | base_url = base_url + '/' 73 | 74 | if result and result.html: 75 | soup = BeautifulSoup(result.html, 'html.parser') 76 | if soup.title: 77 | url_obj.title = soup.title 78 | if self.depth_allowed > depth or self.depth_allowed <= 0: 79 | link_list = [ 80 | soup.find_all('a'), soup.find_all('link'), 81 | soup.find_all('img') 82 | ] 83 | for links in link_list: 84 | for link in links: 85 | if link.name == 'img': 86 | lnk = link.get('src') 87 | else: 88 | lnk = link.get('href') 89 | 90 | if not lnk or lnk == '#': 91 | continue 92 | lnk = self.construct_link(ourl, scheme, netloc, 93 | url, base_url, lnk) 94 | if lnk: 95 | self.crawl_next_link(lnk, session, base_url, 96 | depth, result.out_dir) 97 | 98 | def crawl_next_link(self, lnk, session, base_url, depth, out_dir): 99 | n = urllib.parse.urlparse(lnk) 100 | crawl_allow = False 101 | if len(self.domains_allowed) > 1: 102 | for dm in self.domains_allowed: 103 | if dm in n.netloc or n.netloc == dm: 104 | crawl_allow = True 105 | if not self.crawl_dict.get(lnk) and lnk not in self.link_set: 106 | self.link_set.add(lnk) 107 | if lnk.startswith(base_url) or self.all_domain or crawl_allow: 108 | self.vnt.crawl(lnk, depth=depth+1, session=session, 109 | method='CRAWL_CHILDREN', 110 | crawl_object=self, 111 | onfinished=self.onfinished, 112 | out=out_dir) 113 | 114 | def construct_link(self, ourl, scheme, 115 | netloc, url, base_url, 116 | lnk): 117 | if lnk and '#' in lnk: 118 | pre, ext = lnk.rsplit('#', 1) 119 | if '/' not in ext and pre: 120 | lnk = pre 121 | if lnk and lnk.startswith('//'): 122 | lnk = scheme + ':' + lnk 123 | elif lnk and lnk.startswith('/'): 124 | lnk = lnk[1:] 125 | lnk = scheme+ '://' + netloc + '/' + lnk 126 | elif lnk and lnk.startswith('./'): 127 | lnk = lnk[2:] 128 | lnk = base_url + lnk 129 | elif lnk and lnk.startswith('../'): 130 | lnk = lnk[3:] 131 | lnk = url.rsplit('/', 2)[0] + '/' + lnk 132 | elif lnk and lnk.startswith('#'): 133 | lnk = url 134 | elif lnk and not lnk.startswith('http'): 135 | lnk = base_url + lnk 136 | return lnk 137 | -------------------------------------------------------------------------------- /vinanti/formdata.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of vinanti. 5 | 6 | vinanti is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Lesser General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | vinanti is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Lesser General Public License for more details. 15 | 16 | You should have received a copy of the GNU Lesser General Public License 17 | along with vinanti. If not, see . 18 | """ 19 | 20 | import os 21 | import uuid 22 | import mimetypes 23 | 24 | class Formdata: 25 | 26 | def __init__(self, form_dict, file_dict): 27 | self.form_dict = form_dict 28 | self.file_dict = file_dict 29 | self.final_list = [] 30 | boundary = str(uuid.uuid4()) 31 | boundary = boundary.replace('-', '') 32 | self.boundary = '----------' + boundary 33 | 34 | def get_content_type(self, filename): 35 | return mimetypes.guess_type (filename)[0] or 'application/octet-stream' 36 | 37 | def arrange_files(self, file_title, file_path, boundary, new_boundary=None): 38 | file_type = self.get_content_type(file_path) 39 | file_name = os.path.basename(file_path) 40 | if new_boundary: 41 | self.final_list.append(bytes(new_boundary, 'utf-8')) 42 | else: 43 | self.final_list.append(bytes(boundary, 'utf-8')) 44 | if new_boundary: 45 | hdr = 'Content-Disposition: file; filename="{}"'.format('files', file_name) 46 | else: 47 | hdr = 'Content-Disposition: form-data; name="{}"; filename="{}"'.format(file_title, file_name) 48 | self.final_list.append(bytes(hdr, 'utf-8')) 49 | hdr = 'Content-Type: {}'.format(file_type) 50 | self.final_list.append(bytes(hdr, 'utf-8')) 51 | self.final_list.append(b'') 52 | with open(file_path, 'rb') as f: 53 | content = f.read() 54 | self.final_list.append(content) 55 | 56 | def create_content(self): 57 | boundary = '--' + self.boundary 58 | if isinstance(self.form_dict, (dict, tuple)): 59 | for key_val in self.form_dict: 60 | if isinstance(self.form_dict, dict): 61 | key = key_val 62 | value = self.form_dict.get(key) 63 | else: 64 | key, value = key_val 65 | self.final_list.append(bytes(boundary, 'utf-8')) 66 | hdr = 'Content-Disposition: form-data; name="{}"'.format(key) 67 | self.final_list.append(bytes(hdr, 'utf-8')) 68 | self.final_list.append(b'') 69 | self.final_list.append(bytes(value, 'utf-8')) 70 | if self.file_dict and isinstance(self.file_dict, str): 71 | self.arrange_files('filedata', self.file_dict, boundary) 72 | elif self.file_dict and isinstance(self.file_dict, tuple): 73 | for i, value in enumerate(self.file_dict): 74 | title = 'filedata-{}'.format(i) 75 | self.arrange_files(title, value, boundary) 76 | elif self.file_dict and isinstance(self.file_dict, dict): 77 | for key, value in self.file_dict.items(): 78 | self.arrange_files(key, value, boundary) 79 | self.final_list.append(bytes(boundary+'--', 'utf-8')) 80 | self.final_list.append(b'') 81 | body = b'\r\n'.join (self.final_list) 82 | hdrs = { 83 | 'Content-Type': 'multipart/form-data; boundary={}'.format(self.boundary), 84 | 'Content-Length': str(len(body)) 85 | } 86 | return body, hdrs 87 | 88 | -------------------------------------------------------------------------------- /vinanti/log.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of vinanti. 5 | 6 | vinanti is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Lesser General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | vinanti is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Lesser General Public License for more details. 15 | 16 | You should have received a copy of the GNU Lesser General Public License 17 | along with vinanti. If not, see . 18 | """ 19 | 20 | import logging 21 | 22 | def log_function(name): 23 | logging.basicConfig(level=logging.DEBUG) 24 | logging.getLogger('asyncio').setLevel(logging.WARNING) 25 | #fmt = '%(asctime)-15s::%(module)s:%(funcName)s: %(levelname)-7s - %(message)s' 26 | #formatter_ch = logging.Formatter(fmt) 27 | fmt = '%(lineno)s::%(levelname)s::%(module)s::%(funcName)s: %(message)s' 28 | formatter_ch = logging.Formatter(fmt) 29 | ch = logging.StreamHandler() 30 | ch.setLevel(logging.DEBUG) 31 | ch.setFormatter(formatter_ch) 32 | logger = logging.getLogger(name) 33 | logger.addHandler(ch) 34 | return logger 35 | -------------------------------------------------------------------------------- /vinanti/req.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of vinanti. 5 | 6 | vinanti is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Lesser General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | vinanti is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Lesser General Public License for more details. 15 | 16 | You should have received a copy of the GNU Lesser General Public License 17 | along with vinanti. If not, see . 18 | """ 19 | import os 20 | import urllib.parse 21 | 22 | try: 23 | from vinanti.log import log_function 24 | from vinanti.formdata import Formdata 25 | except ImportError: 26 | from log import log_function 27 | from formdata import Formdata 28 | 29 | logger = log_function(__name__) 30 | 31 | class RequestObject: 32 | 33 | def __init__(self, url, hdrs, method, backend, kargs): 34 | self.url = url 35 | self.hdrs = hdrs 36 | self.kargs = kargs 37 | self.html = None 38 | self.status = None 39 | self.info = None 40 | self.method = method 41 | self.error = None 42 | self.data = None 43 | self.backend = backend 44 | self.log = kargs.get('log') 45 | self.wait = kargs.get('wait') 46 | self.proxies = kargs.get('proxies') 47 | self.auth = kargs.get('auth') 48 | self.auth_digest = kargs.get('auth_digest') 49 | self.files = kargs.get('files') 50 | self.binary = kargs.get('binary') 51 | self.charset = kargs.get('charset') 52 | self.session = kargs.get('session') 53 | self.verify = kargs.get('verify') 54 | if not self.log: 55 | logger.disabled = True 56 | self.timeout = self.kargs.get('timeout') 57 | self.out = self.kargs.get('out') 58 | self.out_dir = None 59 | self.continue_out = self.kargs.get('continue_out') 60 | self.__init_extra__() 61 | 62 | def __init_extra__(self): 63 | self.data_old = None 64 | if self.out: 65 | path_name = self.url.rsplit('/', 1)[-1] 66 | if self.out == 'default' and path_name: 67 | self.out = path_name 68 | elif os.path.isdir(self.out) and path_name: 69 | self.out_dir = self.out 70 | self.out = os.path.join(self.out, path_name) 71 | if os.path.isfile(self.out) and self.continue_out: 72 | sz = os.stat(self.out).st_size 73 | self.hdrs.update({'Range':'bytes={}-'.format(sz)}) 74 | if not self.hdrs: 75 | self.hdrs = {"User-Agent":"Mozilla/5.0"} 76 | if not self.method: 77 | self.method = 'GET' 78 | if not self.timeout: 79 | self.timeout = None 80 | if self.method in ['POST', 'PUT', 'DELETE', 'PATCH']: 81 | self.data = self.kargs.get('data') 82 | if self.data: 83 | self.data_old = self.data 84 | if self.backend == 'urllib': 85 | self.data = urllib.parse.urlencode(self.data) 86 | self.data = self.data.encode('utf-8') 87 | elif self.method == 'GET': 88 | payload = self.kargs.get('params') 89 | if payload: 90 | payload = urllib.parse.urlencode(payload) 91 | self.url = self.url + '?' + payload 92 | if self.files and self.backend == 'urllib': 93 | if self.data: 94 | mfiles = Formdata(self.data_old, self.files) 95 | else: 96 | mfiles = Formdata({}, self.files) 97 | data, hdr = mfiles.create_content() 98 | for key, value in hdr.items(): 99 | self.hdrs.update({key:value}) 100 | self.data = data 101 | 102 | 103 | class Response: 104 | 105 | def __init__(self, url, method=None, error=None, 106 | session_cookies=None, charset=None, 107 | info=None, status=None, content_type=None, 108 | content_encoding=None, html=None, 109 | out_file=None, out_dir=None, binary=None): 110 | self.method = method 111 | self.error = error 112 | self.session_cookies = session_cookies 113 | self.charset = charset 114 | self.html = html 115 | self.info = info 116 | self.status = status 117 | self.url = url 118 | self.content_type = content_type 119 | self.content_encoding = content_encoding 120 | self.out_file = out_file 121 | self.out_dir = out_dir 122 | self.binary = binary 123 | self.request_object = None 124 | self.dstorage = None 125 | -------------------------------------------------------------------------------- /vinanti/req_aio.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of vinanti. 5 | 6 | vinanti is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Lesser General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | vinanti is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Lesser General Public License for more details. 15 | 16 | You should have received a copy of the GNU Lesser General Public License 17 | along with vinanti. If not, see . 18 | """ 19 | 20 | import os 21 | import sys 22 | 23 | try: 24 | import aiohttp 25 | except ImportError: 26 | pass 27 | 28 | import asyncio 29 | import mimetypes 30 | 31 | try: 32 | from vinanti.req import * 33 | from vinanti.log import log_function 34 | except ImportError: 35 | from req import * 36 | from log import log_function 37 | 38 | logger = log_function(__name__) 39 | 40 | 41 | class RequestObjectAiohttp(RequestObject): 42 | 43 | def __init__(self, url, hdrs, method, kargs): 44 | super().__init__(url, hdrs, method, 'aiohttp', kargs) 45 | self.readable_format = [ 46 | 'text/plain', 'text/html', 'text/css', 47 | 'text/javascript', 'application/xhtml+xml', 48 | 'application/xml', 'application/json', 49 | 'application/javascript', 'application/ecmascript' 50 | ] 51 | 52 | async def process_aio_request(self, session): 53 | 54 | func = self.get_aio_request_func(session) 55 | ret_obj = None 56 | async with func as resp: 57 | rsp = Response(self.url, method=self.method, 58 | out_file=self.out, out_dir=self.out_dir) 59 | rsp.info = resp.headers 60 | rsp.content_type = rsp.info.get('content-type') 61 | sz = rsp.info.get('content-length') 62 | rsp.status = resp.status 63 | if sz: 64 | sz = round(int(sz)/(1024*1024), 2) 65 | if rsp.status in [200, 206]: 66 | rsp.url = str(resp.url) 67 | path_name = rsp.url.rsplit('/', 1)[-1] 68 | human_readable = False 69 | for i in self.readable_format: 70 | if i in rsp.content_type.lower(): 71 | human_readable = True 72 | break 73 | text = None 74 | if self.method != 'HEAD': 75 | if self.out: 76 | print_count = 0 77 | if self.continue_out: 78 | mode = 'ab' 79 | else: 80 | mode = 'wb' 81 | with open(self.out, mode) as fd: 82 | while True: 83 | chunk = await resp.content.read(1024) 84 | if not chunk: 85 | break 86 | fd.write(chunk) 87 | print_count += 1 88 | if (print_count) % 200 == 0: 89 | count = print_count * len(chunk) 90 | dwn = round(int(count)/(1024*1024), 2) 91 | sys.stdout.write('\r') 92 | sys.stdout.write('{} M / {} M : {}'.format(dwn, sz, self.out)) 93 | sys.stdout.flush() 94 | sys.stdout.write('\r') 95 | sys.stdout.write('{} M / {} M : {}'.format(sz, sz, self.out)) 96 | sys.stdout.flush() 97 | text = 'file saved to:: {}'.format(self.out) 98 | if not human_readable: 99 | rsp.binary = True 100 | elif self.binary: 101 | text = await resp.read() 102 | elif self.charset and human_readable: 103 | text = await resp.text(encoding=self.charset) 104 | elif human_readable: 105 | text = await resp.text(encoding='utf-8') 106 | else: 107 | text = 'Content {} not human readable.'.format(rsp.content_type) 108 | rsp.html = text 109 | rsp.status = resp.status 110 | cj_arr = [] 111 | for c in session.cookie_jar: 112 | cj_arr.append('{}={}'.format(c.key, c.value)) 113 | rsp.session_cookies = ';'.join(cj_arr) 114 | return rsp 115 | 116 | def get_content_type(self, filename): 117 | return mimetypes.guess_type(filename)[0] or 'application/octet-stream' 118 | 119 | def add_formfields(self): 120 | self.data = aiohttp.FormData() 121 | if isinstance(self.data_old, dict): 122 | for key, value in self.data_old.items(): 123 | self.data.add_field(key, value) 124 | elif isinstance(self.data_old, tuple): 125 | for td in self.data_old: 126 | if isinstance(td, tuple): 127 | self.data.add_field(td[0], td[1]) 128 | if isinstance(self.files, str): 129 | content_type = self.get_content_type(self.files) 130 | filename = os.path.basename(self.files) 131 | self.data.add_field(filename, open(self.files, 'rb'), 132 | content_type=content_type) 133 | elif isinstance(self.files, tuple): 134 | for file_name in self.files: 135 | content_type = self.get_content_type(file_name) 136 | filename = os.path.basename(file_name) 137 | self.data.add_field(filename, open(file_name, 'rb'), 138 | content_type=content_type) 139 | elif isinstance(self.files, dict): 140 | for file_title, file_name in self.files.items(): 141 | content_type = self.get_content_type(file_name) 142 | self.data.add_field(file_title, open(file_name, 'rb'), 143 | content_type=content_type) 144 | 145 | def get_aio_request_func(self, session): 146 | if self.files: 147 | self.add_formfields() 148 | if self.method == 'GET': 149 | func = session.get 150 | elif self.method == 'POST': 151 | func = session.post 152 | elif self.method == 'PUT': 153 | func = session.put 154 | elif self.method == 'PATCH': 155 | func = session.patch 156 | elif self.method == 'DELETE': 157 | func = session.delete 158 | elif self.method == 'HEAD': 159 | func = session.head 160 | elif self.method == 'OPTIONS': 161 | func = session.options 162 | if self.timeout is None: 163 | self.timeout = 300 164 | if self.verify is False: 165 | verify = False 166 | else: 167 | verify = True 168 | http_proxy = None 169 | if self.proxies: 170 | http_proxy = self.proxies.get('http') 171 | if not http_proxy: 172 | http_proxy = self.proxies.get('https') 173 | new_func = func(self.url, headers=self.hdrs, timeout=self.timeout, 174 | ssl=verify, proxy=http_proxy, data=self.data) 175 | 176 | return new_func 177 | 178 | -------------------------------------------------------------------------------- /vinanti/req_urllib.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of vinanti. 5 | 6 | vinanti is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Lesser General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | vinanti is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Lesser General Public License for more details. 15 | 16 | You should have received a copy of the GNU Lesser General Public License 17 | along with vinanti. If not, see . 18 | """ 19 | 20 | import re 21 | import ssl 22 | import gzip 23 | import time 24 | import shutil 25 | import base64 26 | import urllib.parse 27 | import urllib.request 28 | import http.cookiejar 29 | from io import StringIO, BytesIO 30 | 31 | try: 32 | from vinanti.req import * 33 | from vinanti.log import log_function 34 | except ImportError: 35 | from req import * 36 | from log import log_function 37 | 38 | logger = log_function(__name__) 39 | 40 | class RequestObjectUrllib(RequestObject): 41 | 42 | def __init__(self, url, hdrs, method, kargs): 43 | super().__init__(url, hdrs, method, 'urllib', kargs) 44 | 45 | def process_request(self): 46 | opener = None 47 | cj = None 48 | if self.verify is False: 49 | opener = self.handle_https_context(opener, False) 50 | if self.proxies: 51 | opener = self.add_proxy(opener) 52 | if self.session: 53 | opener, cj = self.enable_cookies(opener) 54 | 55 | req = urllib.request.Request(self.url, data=self.data, 56 | headers=self.hdrs, 57 | method=self.method) 58 | if self.auth: 59 | opener = self.add_http_auth(self.auth, 'basic', opener) 60 | elif self.auth_digest: 61 | opener = self.add_http_auth(self.auth_digest, 'digest', opener) 62 | try: 63 | if opener: 64 | r_open = opener.open(req, timeout=self.timeout) 65 | else: 66 | r_open = urllib.request.urlopen(req, timeout=self.timeout) 67 | except Exception as err: 68 | r_open = None 69 | self.error = str(err) 70 | logger.error(err) 71 | ret_obj = ResponseUrllib(self, r_open, cj) 72 | return ret_obj 73 | 74 | def add_http_auth(self, auth_tuple, auth_type, opener=None): 75 | logger.info(auth_type) 76 | usr = auth_tuple[0] 77 | passwd = auth_tuple[1] 78 | if len(auth_tuple) == 2: 79 | realm = None 80 | elif len(auth_tuple) == 3: 81 | realm = auth_tuple[2] 82 | password_manager = urllib.request.HTTPPasswordMgrWithDefaultRealm() 83 | password_manager.add_password(realm, self.url, usr, passwd) 84 | if auth_type == 'basic': 85 | auth_handler = urllib.request.HTTPBasicAuthHandler(password_manager) 86 | else: 87 | auth_handler = urllib.request.HTTPDigestAuthHandler(password_manager) 88 | if opener: 89 | logger.info('Adding Handle to Existing Opener') 90 | opener.add_handler(auth_handler) 91 | else: 92 | opener = urllib.request.build_opener(auth_handler) 93 | return opener 94 | """ 95 | credentials = '{}:{}'.format(usr, passwd) 96 | encoded_credentials = base64.b64encode(bytes(credentials, 'utf-8')) 97 | req.add_header('Authorization', 'Basic {}'.format(encoded_credentials.decode('utf-8'))) 98 | return req 99 | """ 100 | 101 | def handle_https_context(self, opener, verify): 102 | context = ssl.create_default_context() 103 | if verify is False: 104 | context.check_hostname = False 105 | context.verify_mode = ssl.CERT_NONE 106 | https_handler = urllib.request.HTTPSHandler(context=context) 107 | if opener: 108 | logger.info('Adding HTTPS Handle to Existing Opener') 109 | opener.add_handler(https_handler) 110 | else: 111 | opener = urllib.request.build_opener(https_handler) 112 | return opener 113 | 114 | def enable_cookies(self, opener): 115 | cj = http.cookiejar.CookieJar() 116 | cookie_handler = urllib.request.HTTPCookieProcessor(cj) 117 | if opener: 118 | logger.info('Adding Cookie Handle to Existing Opener') 119 | opener.add_handler(cookie_handler) 120 | else: 121 | opener = urllib.request.build_opener(cookie_handler) 122 | return opener, cj 123 | 124 | def add_proxy(self, opener): 125 | logger.info('proxies {}'.format(self.proxies)) 126 | proxy_handler = urllib.request.ProxyHandler(self.proxies) 127 | if opener: 128 | logger.info('Adding Proxy Handle to Existing Opener') 129 | opener.add_handler(proxy_handler) 130 | else: 131 | opener = urllib.request.build_opener(proxy_handler) 132 | return opener 133 | 134 | 135 | class ResponseUrllib(Response): 136 | 137 | def __init__(self, parent=None, req=None, cj=None): 138 | super().__init__(parent.url, error=parent.error, 139 | method=parent.method, out_file=parent.out, 140 | out_dir=parent.out_dir) 141 | if req: 142 | self.request_object = req 143 | self.set_information(req, parent) 144 | self.set_session_cookies(cj) 145 | 146 | def set_information(self, req, parent): 147 | self.info = req.info() 148 | self.url = req.geturl() 149 | self.status = req.getcode() 150 | self.content_encoding = self.info.get('content-encoding') 151 | self.content_type = self.info.get('content-type') 152 | 153 | if not self.content_type: 154 | self.content_type = 'Not Available' 155 | else: 156 | charset_s = re.search('charset[^;]*', self.content_type.lower()) 157 | if charset_s: 158 | charset_t = charset_s.group() 159 | charset_t = charset_t.replace('charset=', '') 160 | self.charset = charset_t.strip() 161 | if parent.charset: 162 | self.charset = parent.charset 163 | 164 | self.readable_format = [ 165 | 'text/plain', 'text/html', 'text/css', 'text/javascript', 166 | 'application/xhtml+xml', 'application/xml', 'application/json', 167 | 'application/javascript', 'application/ecmascript' 168 | ] 169 | human_readable = False 170 | for i in self.readable_format: 171 | if i in self.content_type.lower(): 172 | human_readable = True 173 | break 174 | if not human_readable: 175 | self.binary = True 176 | dstorage = None 177 | if self.content_encoding == 'gzip': 178 | try: 179 | storage = BytesIO(req.read()) 180 | dstorage = gzip.GzipFile(fileobj=storage) 181 | except Exception as err: 182 | logger.error(err) 183 | self.dstorage = dstorage 184 | if parent.method == 'HEAD': 185 | self.html = 'None' 186 | elif parent.out: 187 | if parent.continue_out: 188 | mode = 'ab' 189 | else: 190 | mode = 'wb' 191 | with open(parent.out, mode) as out_file: 192 | if dstorage is None: 193 | shutil.copyfileobj(req, out_file) 194 | else: 195 | shutil.copyfileobj(dstorage, out_file) 196 | self.html = 'file saved to:: {}'.format(parent.out) 197 | else: 198 | self.read_html(parent, req, dstorage, human_readable) 199 | 200 | def save(self, req, out_file, continue_out=False): 201 | mode = 'wb' 202 | if continue_out: 203 | mode = 'ab' 204 | if req: 205 | with open(out_file, mode) as out_file: 206 | if self.dstorage is None: 207 | shutil.copyfileobj(req, out_file) 208 | else: 209 | shutil.copyfileobj(dstorage, out_file) 210 | 211 | def read_html(self, parent, req, dstorage, human_readable): 212 | try: 213 | decoding_required = False 214 | if dstorage is None and human_readable and not parent.binary: 215 | self.html = req.read() 216 | decoding_required = True 217 | elif dstorage and human_readable and not parent.binary: 218 | self.html = dstorage.read() 219 | decoding_required = True 220 | elif parent.binary: 221 | self.html = req.read() 222 | else: 223 | self.html = ('not human readable content: content-type is {}' 224 | .format(self.content_type)) 225 | if decoding_required: 226 | if self.charset: 227 | try: 228 | self.html = self.html.decode(self.charset) 229 | except Exception as err: 230 | logger.error(err) 231 | self.html = self.html.decode('utf-8') 232 | else: 233 | self.html = self.html.decode('utf-8') 234 | except Exception as err: 235 | logger.error(err) 236 | self.html = str(err) 237 | 238 | def set_session_cookies(self, cj): 239 | if cj: 240 | cj_arr = [] 241 | for i in cj: 242 | cj_arr.append('{}={}'.format(i.name, i.value)) 243 | self.session_cookies = ';'.join(cj_arr) 244 | else: 245 | for i in self.info.walk(): 246 | cookie_list = i.get_all('set-cookie') 247 | cookie_jar = [] 248 | if cookie_list: 249 | for i in cookie_list: 250 | cookie = i.split(';')[0] 251 | cookie_jar.append(cookie) 252 | if cookie_jar: 253 | cookies = ';'.join(cookie_jar) 254 | self.session_cookies = cookies 255 | -------------------------------------------------------------------------------- /vinanti/utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (C) 2018 kanishka-linux kanishka.linux@gmail.com 3 | 4 | This file is part of vinanti. 5 | 6 | vinanti is free software: you can redistribute it and/or modify 7 | it under the terms of the GNU Lesser General Public License as published by 8 | the Free Software Foundation, either version 3 of the License, or 9 | (at your option) any later version. 10 | 11 | vinanti is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU Lesser General Public License for more details. 15 | 16 | You should have received a copy of the GNU Lesser General Public License 17 | along with vinanti. If not, see . 18 | """ 19 | 20 | try: 21 | from vinanti.req_urllib import RequestObjectUrllib 22 | from vinanti.log import log_function 23 | except ImportError: 24 | from req_urllib import RequestObjectUrllib 25 | from log import log_function 26 | 27 | logger = log_function(__name__) 28 | 29 | 30 | def complete_function_request(func, kargs): 31 | req_obj = func(*kargs) 32 | return req_obj 33 | 34 | 35 | def get_request(backend, url, hdrs, method, kargs): 36 | req_obj = None 37 | if backend == 'urllib': 38 | req = RequestObjectUrllib(url, hdrs, method, kargs) 39 | req_obj = req.process_request() 40 | return req_obj 41 | 42 | 43 | class URL: 44 | 45 | def __init__(self, url, depth=0): 46 | self.url = url 47 | self.depth = depth 48 | self.title = '' 49 | --------------------------------------------------------------------------------