├── .gitignore ├── LICENSE ├── Procfile ├── README.md ├── fakedatagenerator ├── db.sqlite3 ├── fakedatagenerator │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py ├── generator │ ├── __init__.py │ ├── admin.py │ ├── apps.py │ ├── constants.py │ ├── forms.py │ ├── migrations │ │ └── __init__.py │ ├── models.py │ ├── templates │ │ └── generator │ │ │ └── index.html │ ├── tests.py │ ├── urls.py │ ├── utils.py │ └── views.py └── manage.py ├── requirements.txt └── runtime.txt /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | 27 | # PyInstaller 28 | # Usually these files are written by a python script from a template 29 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 30 | *.manifest 31 | *.spec 32 | 33 | # Installer logs 34 | pip-log.txt 35 | pip-delete-this-directory.txt 36 | 37 | # Unit test / coverage reports 38 | htmlcov/ 39 | .tox/ 40 | .coverage 41 | .coverage.* 42 | .cache 43 | nosetests.xml 44 | coverage.xml 45 | *,cover 46 | .hypothesis/ 47 | 48 | # Translations 49 | *.mo 50 | *.pot 51 | 52 | # Django stuff: 53 | *.log 54 | local_settings.py 55 | 56 | # Flask stuff: 57 | instance/ 58 | .webassets-cache 59 | 60 | # Scrapy stuff: 61 | .scrapy 62 | 63 | # Sphinx documentation 64 | docs/_build/ 65 | 66 | # PyBuilder 67 | target/ 68 | 69 | # IPython Notebook 70 | .ipynb_checkpoints 71 | 72 | # pyenv 73 | .python-version 74 | 75 | # celery beat schedule file 76 | celerybeat-schedule 77 | 78 | # dotenv 79 | .env 80 | 81 | # virtualenv 82 | venv/ 83 | ENV/ 84 | 85 | # Spyder project settings 86 | .spyderproject 87 | 88 | # Rope project settings 89 | .ropeproject 90 | 91 | # Notepad++ 92 | *.bak 93 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Procfile: -------------------------------------------------------------------------------- 1 | web: python fakedatagenerator/manage.py runserver 0.0.0.0:$PORT -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # FakeDataGenerator 2 | 3 | ## Overview 4 | FakeDataGenerator is a Django web app that let's users create fake datasets for their data science/machine learning projects. Lately, I've been trying to learn machine learning and implement basic 1D versions of popular algorithms. However, I couldn't find anything online that just gave me a simple txt file or csv file. So, I decided to create FakeDataGenerator. 5 | 6 | ## What I've learned 7 | I haven't learned a lot during this project but it has increased my understanding on what I can and cannot do with the Django framework. I learned how to easily incoporate downloads into a django web app (in less than 5 lines of code). I have also learned how to deal with csv files. Also, I honestly didn't think I would be able to finish such a project in so little time. It is nice to see how my skills with the Django web framework have developped over the past year and I hope to improve over the next years. 8 | 9 | ## Improvements 10 | As always, there is a ton that can be done to improve this project. A major thing that needs to be improved it the formatting of the text file when a user downloads a dataset. When I run the standalone generate function, the text file has the perfect formatting. However, this is not the case when one downloads a file straight from the website. And as with my other projects, both my code design and web design skills need to be improved. 11 | -------------------------------------------------------------------------------- /fakedatagenerator/db.sqlite3: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mikerah/FakeDataGenerator/fa6c3d0c01f060b69ab27d0402ff500c75d19165/fakedatagenerator/db.sqlite3 -------------------------------------------------------------------------------- /fakedatagenerator/fakedatagenerator/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mikerah/FakeDataGenerator/fa6c3d0c01f060b69ab27d0402ff500c75d19165/fakedatagenerator/fakedatagenerator/__init__.py -------------------------------------------------------------------------------- /fakedatagenerator/fakedatagenerator/settings.py: -------------------------------------------------------------------------------- 1 | """ 2 | Django settings for fakedatagenerator project. 3 | 4 | Generated by 'django-admin startproject' using Django 1.10.5. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/1.10/topics/settings/ 8 | 9 | For the full list of settings and their values, see 10 | https://docs.djangoproject.com/en/1.10/ref/settings/ 11 | """ 12 | 13 | import os 14 | 15 | # Build paths inside the project like this: os.path.join(BASE_DIR, ...) 16 | BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 17 | PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__)) 18 | 19 | 20 | # Quick-start development settings - unsuitable for production 21 | # See https://docs.djangoproject.com/en/1.10/howto/deployment/checklist/ 22 | 23 | # SECURITY WARNING: keep the secret key used in production secret! 24 | SECRET_KEY = 'gvzk6)7al&obqdy1)#)ztxcvg_@)mh!i2(ff*4^%f29zhq_w0-' 25 | 26 | # SECURITY WARNING: don't run with debug turned on in production! 27 | DEBUG = True 28 | 29 | ALLOWED_HOSTS = ['*'] 30 | 31 | 32 | # Application definition 33 | 34 | INSTALLED_APPS = [ 35 | 'django.contrib.admin', 36 | 'django.contrib.auth', 37 | 'django.contrib.contenttypes', 38 | 'django.contrib.sessions', 39 | 'django.contrib.messages', 40 | 'django.contrib.staticfiles', 41 | 'generator', 42 | 'bootstrap3' 43 | ] 44 | 45 | MIDDLEWARE = [ 46 | 'django.middleware.security.SecurityMiddleware', 47 | 'django.contrib.sessions.middleware.SessionMiddleware', 48 | 'django.middleware.common.CommonMiddleware', 49 | 'django.middleware.csrf.CsrfViewMiddleware', 50 | 'django.contrib.auth.middleware.AuthenticationMiddleware', 51 | 'django.contrib.messages.middleware.MessageMiddleware', 52 | 'django.middleware.clickjacking.XFrameOptionsMiddleware', 53 | ] 54 | 55 | ROOT_URLCONF = 'fakedatagenerator.urls' 56 | 57 | TEMPLATES = [ 58 | { 59 | 'BACKEND': 'django.template.backends.django.DjangoTemplates', 60 | 'DIRS': [], 61 | 'APP_DIRS': True, 62 | 'OPTIONS': { 63 | 'context_processors': [ 64 | 'django.template.context_processors.debug', 65 | 'django.template.context_processors.request', 66 | 'django.contrib.auth.context_processors.auth', 67 | 'django.contrib.messages.context_processors.messages', 68 | ], 69 | }, 70 | }, 71 | ] 72 | 73 | WSGI_APPLICATION = 'fakedatagenerator.wsgi.application' 74 | 75 | 76 | # Database 77 | # https://docs.djangoproject.com/en/1.10/ref/settings/#databases 78 | 79 | DATABASES = { 80 | 81 | 'default': { 82 | 'ENGINE': 'django.db.backends.sqlite3', 83 | 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), 84 | } 85 | 86 | } 87 | 88 | 89 | # Password validation 90 | # https://docs.djangoproject.com/en/1.10/ref/settings/#auth-password-validators 91 | 92 | AUTH_PASSWORD_VALIDATORS = [ 93 | { 94 | 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', 95 | }, 96 | { 97 | 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', 98 | }, 99 | { 100 | 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', 101 | }, 102 | { 103 | 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', 104 | }, 105 | ] 106 | 107 | 108 | # Internationalization 109 | # https://docs.djangoproject.com/en/1.10/topics/i18n/ 110 | 111 | LANGUAGE_CODE = 'en-us' 112 | 113 | TIME_ZONE = 'UTC' 114 | 115 | USE_I18N = True 116 | 117 | USE_L10N = True 118 | 119 | USE_TZ = True 120 | 121 | 122 | # Static files (CSS, JavaScript, Images) 123 | # https://docs.djangoproject.com/en/1.10/howto/static-files/ 124 | 125 | STATIC_URL = '/static/' 126 | PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__)) 127 | STATIC_ROOT = os.path.join(PROJECT_ROOT, 'static') 128 | -------------------------------------------------------------------------------- /fakedatagenerator/fakedatagenerator/urls.py: -------------------------------------------------------------------------------- 1 | """fakedatagenerator URL Configuration 2 | 3 | The `urlpatterns` list routes URLs to views. For more information please see: 4 | https://docs.djangoproject.com/en/1.10/topics/http/urls/ 5 | Examples: 6 | Function views 7 | 1. Add an import: from my_app import views 8 | 2. Add a URL to urlpatterns: url(r'^$', views.home, name='home') 9 | Class-based views 10 | 1. Add an import: from other_app.views import Home 11 | 2. Add a URL to urlpatterns: url(r'^$', Home.as_view(), name='home') 12 | Including another URLconf 13 | 1. Import the include() function: from django.conf.urls import url, include 14 | 2. Add a URL to urlpatterns: url(r'^blog/', include('blog.urls')) 15 | """ 16 | from django.conf.urls import url, include 17 | from django.contrib import admin 18 | 19 | urlpatterns = [ 20 | url(r'^', include('generator.urls')), 21 | url(r'^admin/', admin.site.urls), 22 | ] 23 | -------------------------------------------------------------------------------- /fakedatagenerator/fakedatagenerator/wsgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | WSGI config for fakedatagenerator project. 3 | 4 | It exposes the WSGI callable as a module-level variable named ``application``. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/1.10/howto/deployment/wsgi/ 8 | """ 9 | 10 | import os 11 | 12 | from django.core.wsgi import get_wsgi_application 13 | 14 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "fakedatagenerator.settings") 15 | 16 | application = get_wsgi_application() 17 | -------------------------------------------------------------------------------- /fakedatagenerator/generator/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mikerah/FakeDataGenerator/fa6c3d0c01f060b69ab27d0402ff500c75d19165/fakedatagenerator/generator/__init__.py -------------------------------------------------------------------------------- /fakedatagenerator/generator/admin.py: -------------------------------------------------------------------------------- 1 | from django.contrib import admin 2 | 3 | # Register your models here. 4 | -------------------------------------------------------------------------------- /fakedatagenerator/generator/apps.py: -------------------------------------------------------------------------------- 1 | from django.apps import AppConfig 2 | 3 | 4 | class GeneratorConfig(AppConfig): 5 | name = 'generator' 6 | -------------------------------------------------------------------------------- /fakedatagenerator/generator/constants.py: -------------------------------------------------------------------------------- 1 | # Contains constants used in views.py 2 | 3 | types = {1: "regression", 2: "classification"} 4 | file_formats = {1: "txt", 2: "csv", 3: "json"} -------------------------------------------------------------------------------- /fakedatagenerator/generator/forms.py: -------------------------------------------------------------------------------- 1 | from django import forms 2 | 3 | class QueryForm(forms.Form): 4 | type = forms.ChoiceField(choices=[(1,"Regression"), (2,"Classification")], required=True) 5 | number_of_predictors = forms.IntegerField(max_value=20, initial=1, required=True) 6 | number_of_data_points = forms.IntegerField(max_value=10000, initial=100, required=True) 7 | file_name = forms.CharField(max_length=100, required=True) 8 | file_format = forms.ChoiceField(choices=[(1,"txt"),(2,"csv"), (3, "json")], required=True) -------------------------------------------------------------------------------- /fakedatagenerator/generator/migrations/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mikerah/FakeDataGenerator/fa6c3d0c01f060b69ab27d0402ff500c75d19165/fakedatagenerator/generator/migrations/__init__.py -------------------------------------------------------------------------------- /fakedatagenerator/generator/models.py: -------------------------------------------------------------------------------- 1 | from django.db import models 2 | 3 | # Create your models here. 4 | -------------------------------------------------------------------------------- /fakedatagenerator/generator/templates/generator/index.html: -------------------------------------------------------------------------------- 1 | {% load bootstrap3 %} 2 | {% bootstrap_css %} 3 | 4 | 5 | 6 | 7 | Fake Data Generator 8 | 9 | 10 | 11 | 12 | 13 |

Fake Dataset Generator

14 |

Having difficulties finding a simple dataset for your data scientist needs?

15 |

Here, you can generate fake datasets for testing or learning purposes for free.

16 | Fork me on GitHub 17 |
18 | {% csrf_token %} 19 | {{ form }} 20 | 21 |
22 | 23 | -------------------------------------------------------------------------------- /fakedatagenerator/generator/tests.py: -------------------------------------------------------------------------------- 1 | from django.test import TestCase 2 | 3 | # Create your tests here. 4 | -------------------------------------------------------------------------------- /fakedatagenerator/generator/urls.py: -------------------------------------------------------------------------------- 1 | from django.conf.urls import url 2 | from . import views 3 | 4 | urlpatterns = [ 5 | url(r'^$', views.index, name='index'), 6 | ] -------------------------------------------------------------------------------- /fakedatagenerator/generator/utils.py: -------------------------------------------------------------------------------- 1 | from csv import writer 2 | from json import dump 3 | from random import random, choice 4 | 5 | def generate(n_predictors, n_data_points, file_name, task="regression", file_format="txt"): 6 | """ 7 | Creates a fake dataset with the given specs 8 | n_predictors: number of predictors - int 9 | n_data_points: number of observations - int 10 | file_name: name of output file - string 11 | task: regression or classification - string - default: regression 12 | file_format: txt or csv - string - default: txt 13 | return value: txt file, csv file or json file 14 | """ 15 | row_header = ["y"] 16 | row_header.extend(["x"+str(i+1) for i in range(n_predictors)]) 17 | length_of_header = len(row_header) 18 | if file_format == "txt": 19 | file = open(file_name+".txt", "w+") 20 | 21 | file.write(" ".join(row_header) + "\n") 22 | 23 | if task == "regression": 24 | for i in range(n_data_points): 25 | values = [round(random(),2) for i in range(length_of_header)] 26 | file.write(" ".join(map(str, values)) + "\n") 27 | 28 | elif task == "classification": 29 | for i in range(n_data_points): 30 | values = [choice([0,1])] 31 | values.extend([round(random(),2) for i in range(length_of_header-1)]) 32 | file.write(" ".join(map(str, values)) + "\n") 33 | elif file_format == "csv": 34 | file = open(file_name+".csv", "w+") 35 | datawriter = writer(file, lineterminator="\n") 36 | datawriter.writerow(row_header) 37 | if task == "regression": 38 | for i in range(n_data_points): 39 | values = [round(random(),2) for i in range(length_of_header)] 40 | datawriter.writerow(values) 41 | elif task == "classification": 42 | for i in range(n_data_points): 43 | values = [choice([0,1])] 44 | values.extend([round(random(),2) for i in range(length_of_header-1)]) 45 | datawriter.writerow(values) 46 | 47 | elif file_format == "json": 48 | file = open(file_name+".json", "w+") 49 | json_data = {} 50 | for i in range(n_data_points): 51 | if task == "regression": 52 | json_data[i+1] = {"y": round(random(),2)} 53 | elif task == "classification": 54 | json_data[i+1] = {"y": choice([0,1])} 55 | for j in range(n_predictors): 56 | var_name = "x"+str(j) 57 | json_data[i+1][var_name] = round(random(),2) 58 | 59 | dump(json_data, file) 60 | file.close() 61 | return file 62 | -------------------------------------------------------------------------------- /fakedatagenerator/generator/views.py: -------------------------------------------------------------------------------- 1 | from django.shortcuts import render 2 | from django.http import HttpResponse 3 | from .forms import QueryForm 4 | from .utils import generate 5 | from io import StringIO 6 | from . import constants 7 | 8 | def index(request): 9 | if request.method == "POST": 10 | form = QueryForm(request.POST) 11 | 12 | if form.is_valid(): 13 | type = constants.types[int(form.cleaned_data['type'])] 14 | num_predictors = int(form.cleaned_data['number_of_predictors']) 15 | num_data_points = int(form.cleaned_data['number_of_data_points']) 16 | file_name = form.cleaned_data['file_name'] 17 | file_type = constants.file_formats[int(form.cleaned_data['file_format'])] 18 | 19 | f = generate(num_predictors,num_data_points,file_name,type,file_type) 20 | file_to_dl = open(f.name) 21 | response = HttpResponse(file_to_dl, content_type="text/"+file_type) 22 | response['Content-Disposition'] = 'attachment; filename="{}"'.format(file_to_dl.name) 23 | file_to_dl.close() 24 | return response 25 | else: 26 | form = QueryForm() 27 | 28 | return render(request, 'generator/index.html', {'form': form}) 29 | -------------------------------------------------------------------------------- /fakedatagenerator/manage.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import os 3 | import sys 4 | 5 | if __name__ == "__main__": 6 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "fakedatagenerator.settings") 7 | try: 8 | from django.core.management import execute_from_command_line 9 | except ImportError: 10 | # The above import may fail for some other reason. Ensure that the 11 | # issue is really that Django is missing to avoid masking other 12 | # exceptions on Python 2. 13 | try: 14 | import django 15 | except ImportError: 16 | raise ImportError( 17 | "Couldn't import Django. Are you sure it's installed and " 18 | "available on your PYTHONPATH environment variable? Did you " 19 | "forget to activate a virtual environment?" 20 | ) 21 | raise 22 | execute_from_command_line(sys.argv) 23 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mikerah/FakeDataGenerator/fa6c3d0c01f060b69ab27d0402ff500c75d19165/requirements.txt -------------------------------------------------------------------------------- /runtime.txt: -------------------------------------------------------------------------------- 1 | python-3.6.0 --------------------------------------------------------------------------------