├── .gitignore ├── ChangeLog ├── MANIFEST.in ├── README.md ├── license.txt ├── setup.cfg ├── setup.py └── sqlautocode ├── __init__.py ├── config.py ├── constants.py ├── declarative.py ├── formatter.py ├── loader.py ├── main.py ├── tests ├── __init__.py ├── base.py └── test_declarative.py └── util.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *egg* 3 | -------------------------------------------------------------------------------- /ChangeLog: -------------------------------------------------------------------------------- 1 | 0.6 (unreleased) 2 | Some more fixes for Issues 16 to 24 3 | [spamsch] 4 | 5 | Removing support for SA 0.5.x 6 | [spamsch] 7 | 8 | 0.6b1 9 | Fixes for SA 0.6.x 10 | [percious] 11 | 12 | Many other fixes 13 | [percious, spamsch] 14 | 15 | 0.5.3 16 | Added declarative generation 17 | [percious] 18 | 19 | Added tests for declarative generation 20 | [percious] 21 | 22 | Removed monkey-patching for just importing sqlautocode 23 | [percious] 24 | 25 | 26 | 0.5.2 (UNRELEASED) 27 | - Added table prefix and suffix options (by Wayne Wight) 28 | [spamsch] 29 | 30 | - Fixed assignment error in __init__.py (Issue 10) 31 | [spamsch] 32 | 33 | - Added setuptools script for project management. 34 | [jaraco] 35 | 36 | - Fixed issue with sqlalchemy version checking when it contains non-numeric characters (Issue 7). 37 | [jaraco] 38 | 39 | - Implemented workaround for issue 9: schema handling when no schema is specified. 40 | [jaraco] 41 | 42 | 0.5.1 (10.01.2008) 43 | - Table matching is now chattier about misses. 44 | [jek] 45 | 46 | - Table globbing now supports the full glob syntax 47 | [jek] 48 | 49 | - Tables in --tables can be "quot*ed" to escape globbing chars. 50 | [jek] 51 | 52 | - Engine url is no longer an -option (it's required!) 53 | New syntax is 'autocode.py dburl://...' 54 | [jek] 55 | 56 | - Detection of a SQLAlchemy installation is now a little friendlier. 57 | [jek] 58 | 59 | - Can now choose the generated file --encoding. 60 | [jek] 61 | 62 | - Handling of multi-byte and funk$ky schema identifiers is improved 63 | but not yet 100%. 64 | [jek] 65 | 66 | - Fixed example code 67 | [jek] 68 | 69 | - Fix for indexes with funky$column-names 70 | [jek] 71 | 72 | - Indexes now repr like other schema items 73 | [jek] 74 | 75 | - Column types now emit their arguments (e.g. VARCHAR(32)) 76 | [jek] 77 | 78 | - Fixed parse errors in emitted column definitions 79 | [jek] 80 | 81 | - Casting to generic column types is now optional 82 | [jek] 83 | 84 | - Fixed logic when casting to a generic column type 85 | [jek] 86 | 87 | - Fixed Python 2.5 compatibility issue 88 | [jek] 89 | 90 | - Only emitting schema= on Tables when --schema is present 91 | [jek] 92 | 93 | - Fixed index generation 94 | [jek] 95 | 96 | 0.5 (13.11.2007) 97 | 98 | - Fixed example code to use limit() 99 | [spamsch] 100 | 101 | - Fixed formatter to handle types correctly 102 | [spamsch] 103 | 104 | - Adapted to new Index handling 105 | [spamsch] 106 | 107 | - Fixed commandline support for example generation 108 | [spamsch] 109 | 110 | - Added index listing patch for pgsql 111 | [spamsch] 112 | 113 | - Removed custom autoloader stuff 114 | [spamsch] 115 | 116 | - Removed 0.3 specific code 117 | [spamsch] 118 | 119 | - Added support for z3c.sqlalchemy 120 | [spamsch] 121 | 122 | - Refactored some code 123 | [spamsch] 124 | 125 | 0.4 (10.10.2007) 126 | 127 | - Restructured code to make this module more project like 128 | and to ease the code management. 129 | [spamsch] 130 | 131 | - Support for globbing character in table names 132 | [spamsch] 133 | 134 | - More robust argument handling 135 | [spamsch] 136 | 137 | - Making schema and table selection possible 138 | [spamsch] 139 | 140 | - Make autocode generate some example code 141 | [spamsch] 142 | 143 | - Fixed some minor bugs 144 | [spamsch] 145 | 146 | - Add full commandline support and some options 147 | [spamsch] 148 | 149 | 0.3 (25.07.2007) 150 | 151 | - Simplified code a bit 152 | [sdobrev] 153 | 154 | - Added SQLite and PostGres support 155 | [sdobrev] 156 | 157 | 0.2 (17.07.2007) 158 | 159 | - Takes arguments on the command line to select the dburl and 160 | the output destination 161 | [cdevienne] 162 | 163 | - Replace a bunch of database specific types by generic ones. 164 | This is incomplete as it feats only my needs for a mysql to mssql 165 | database conversion. 166 | [cdevienne] 167 | 168 | - Output the indexes and ForeignKeyConstraints (including multi-columns 169 | ones) correctly 170 | [cdevienne] 171 | 172 | 0.1 (24.02.2007) 173 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include setup.py license.txt readme.txt ChangeLog 2 | include sqlautocode/*.py 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | sqlautocode 2 | =========== 3 | 4 | Effort discontinued. Have switched to sqlacodegen which is cleaner, does a better job at generating model, and works using SQLAlchemy >= 0.9 and Python 3. You can view my fork of the project here. 5 | 6 | --- 7 | 8 | Made some modifications to config.py and declarative.py to get object relationships to work. 9 | 10 | Have also fixed issue 42. http://code.google.com/p/sqlautocode/issues/detail?id=42 11 | 12 | Note that sqlautocode doesn't generate relationships for tables where overlapping composite foreign keys exist. 13 | 14 | --- 15 | 16 | http://code.google.com/p/sqlautocode/ 17 | 18 | ORIGINAL README.md 19 | 20 | AutoCode is a flexible tool to autogenerate a model from an existing database. 21 | 22 | This is a slightly different approach to SqlSoup, 23 | that lets you use tables without explicitly defining them. 24 | 25 | Current Maintainer: 26 | 27 | Chris Perkins (percious) 28 | E-mail: chris@percious.com 29 | 30 | Simon Pamies (spamsch) 31 | E-Mail: s.pamies at banality dot de 32 | 33 | Authors: 34 | 35 | Paul Johnson (original author) 36 | 37 | Christophe de Vienne (cdevienne) 38 | E-Mail: cdevienne at gmail dot com 39 | 40 | Svilen Dobrev (sdobrev) 41 | E-Mail: svilen_dobrev at users point sourceforge dot net 42 | 43 | License: 44 | 45 | MIT 46 | see license.txt 47 | 48 | Requirements: 49 | 50 | sqlalchemy 0.6+ 51 | 52 | Documentation: 53 | 54 | Call sqlautocode.py --help for a list of available self explaining options. 55 | 56 | Example: 57 | sqlautocode.py -o model.py -u postgres://postgres:user@password/MyDatabase -s myschema -t Person*,Download 58 | 59 | ToDo: 60 | 61 | + Generate ActiveMapper / Elixir model 62 | 63 | Notes (random): 64 | 65 | ATT: sqlautocode currently does not handle function indexes well. It generates 66 | code not understood by sqlalchemy. 67 | 68 | (old) metadata stuff from: 69 | http://sqlzoo.cn/howto/source/z.dir/tip137084/i12meta.xml 70 | -------------------------------------------------------------------------------- /license.txt: -------------------------------------------------------------------------------- 1 | This is the MIT license: http://www.opensource.org/licenses/mit-license.php 2 | 3 | Copyright (c) 2007 Paul Johnson, Christophe de Vienne, Svilen Dobrev and Simon Pamies. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this 6 | software and associated documentation files (the "Software"), to deal in the Software 7 | without restriction, including without limitation the rights to use, copy, modify, merge, 8 | publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons 9 | to whom the Software is furnished to do so, subject to the following conditions: 10 | 11 | The above copyright notice and this permission notice shall be included in all copies or 12 | substantial portions of the Software. 13 | 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 15 | INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 16 | PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE 17 | FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR 18 | OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 19 | DEALINGS IN THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [aliases] 2 | release = egg_info -rDb "" sdist register upload 3 | 4 | [nosetests] 5 | with-doctest=1 6 | 7 | [egg_info] 8 | 9 | [flake8] 10 | ignore=E501 11 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!python 2 | # -*- coding: UTF-8 -*- 3 | 4 | """ 5 | Setup script for building sqlautocode 6 | """ 7 | 8 | version = '0.6b1' 9 | 10 | from setuptools import setup, find_packages 11 | 12 | setup( 13 | name='sqlautocode', 14 | version=version, 15 | description='AutoCode is a flexible tool to autogenerate a model from an existing database.', 16 | author='Simon Pamies', 17 | author_email='s.pamies@banality.de', 18 | url='http://code.google.com/p/sqlautocode/', 19 | packages=find_packages(exclude=['ez_setup', 'tests']), 20 | zip_safe=True, 21 | license='MIT', 22 | classifiers=[ 23 | "Development Status :: 4 - Beta", 24 | "Intended Audience :: Developers", 25 | "Programming Language :: Python", 26 | ], 27 | entry_points=dict( 28 | console_scripts=[ 29 | 'sqlautocode = sqlautocode.main:main', 30 | ], 31 | ), 32 | install_requires=[ 33 | 'sqlalchemy', 34 | ], 35 | include_package_data=True, 36 | extras_require={ 37 | }, 38 | dependency_links=[ 39 | ], 40 | tests_require=[ 41 | 'nose>=0.10', 42 | ], 43 | test_suite="nose.collector", 44 | ) 45 | -------------------------------------------------------------------------------- /sqlautocode/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ksindi/sqlautocode/5e9eaf4c70ede361de3397e4346165fa403ca9c0/sqlautocode/__init__.py -------------------------------------------------------------------------------- /sqlautocode/config.py: -------------------------------------------------------------------------------- 1 | #!python 2 | # -*- coding: UTF-8 -*- 3 | 4 | import optparse 5 | import os 6 | import sys 7 | import re 8 | 9 | options = None 10 | out = sys.stdout 11 | err = sys.stderr 12 | dburl = None 13 | engine = None 14 | 15 | 16 | # TODO: encoding (default utf-8) 17 | 18 | def create_parser(): 19 | parser = optparse.OptionParser( 20 | """autocode.py [options, ] 21 | Generates Python source code for a given database schema. 22 | 23 | Example: ./autocode.py postgres://user:password@myhost/database -o out.py""") 24 | 25 | parser.add_option( 26 | "-o", "--output", 27 | help="Write to file (default is stdout)", 28 | action="store", dest="output") 29 | 30 | parser.add_option( 31 | "--force", 32 | help="Overwrite Write to file (default is stdout)", 33 | action="store_true", dest="force") 34 | 35 | parser.add_option( 36 | "-s", "--schema", 37 | help="Optional, reflect a non-default schema", 38 | action="callback", callback=_prep_schema, type="string", dest="schema") 39 | 40 | parser.add_option( 41 | "-t", "--tables", 42 | help=("Optional, only reflect this comma-separated list of tables. " 43 | "Wildcarding with '*' is supported, e.g: --tables account_*," 44 | "orders,order_items,*_audit"), 45 | action="callback", callback=_prep_tables, type="string", dest="tables") 46 | 47 | parser.add_option( 48 | "-b", "--table-prefix", 49 | help="Prefix for generated SQLAlchemy Table object names", 50 | action="store", dest="table_prefix") 51 | 52 | parser.add_option( 53 | "-a", "--table-suffix", 54 | help="Suffix for generated SQLAlchemy Table object names", 55 | action="store", dest="table_suffix") 56 | 57 | parser.add_option( 58 | "-i", "--noindexes", "--noindex", 59 | help="Do not emit index information", 60 | action="store_true", dest="noindex") 61 | 62 | parser.add_option( 63 | "-g", "--generic-types", 64 | help="Emit generic ANSI column types instead of database-specific.", 65 | action="store_true", dest="generictypes") 66 | 67 | parser.add_option( 68 | "--encoding", 69 | help="Encoding for output, default utf8", 70 | action="store", dest="encoding") 71 | 72 | parser.add_option( 73 | "-e", "--example", 74 | help="Generate code with examples how to access data", 75 | action="store_true", dest="example") 76 | 77 | parser.add_option( 78 | "-3", "--z3c", 79 | help="Generate code for use with z3c.sqlalchemy", 80 | action="store_true", dest="z3c") 81 | 82 | parser.add_option( 83 | "-d", "--declarative", 84 | help="Generate declarative SA code", 85 | action="store_true", dest="declarative") 86 | 87 | parser.add_option( 88 | "-n", "--interactive", 89 | help="Generate Interactive example in your code.", 90 | action="store_true", dest="interactive") 91 | 92 | parser.set_defaults(tables=[], 93 | encoding='utf-8', 94 | table_prefix='', 95 | table_suffix='') 96 | 97 | return parser 98 | 99 | 100 | def _prep_tables(option, opt_str, value, parser): 101 | if not value: 102 | parser.values.tables = [] 103 | else: 104 | parser.values.tables = [x.strip() 105 | for x in value.split(',') 106 | if x.strip() != ''] 107 | 108 | 109 | def _prep_schema(option, opt_str, value, parser): 110 | # handle multiple schemas on the command line 111 | value = [x.strip() 112 | for x in value.split(',') 113 | if x.strip() != ''] 114 | if len(value) == 1: 115 | parser.values.schema = value[0] 116 | return 117 | parser.values.schema = value 118 | 119 | 120 | def _version_check(parser): 121 | try: 122 | import sqlalchemy 123 | except ImportError, ex: 124 | parser.error("SQLAlchemy version 0.4.0 or higher is required. (%s)" % 125 | (ex)) 126 | 127 | version = getattr(sqlalchemy, '__version__', None) 128 | if version is None: 129 | parser.error("SQLAlchemy version 0.4.0 or higher is required.") 130 | elif version == 'svn': 131 | pass 132 | else: 133 | non_numeric = re.compile('[^0-9]*') 134 | version_info = tuple([int(i) for i in non_numeric.split(version)]) 135 | if version_info < (0, 4): 136 | parser.error("SQLAlchemy version 0.4.0 or higher is required.") 137 | 138 | 139 | def _setup_engine(parser, url): 140 | global engine 141 | 142 | import sqlalchemy 143 | try: 144 | engine = sqlalchemy.create_engine(url) 145 | test = engine.connect() 146 | test.close() 147 | except sqlalchemy.exc.SQLAlchemyError, ex: 148 | parser.error('Could not connect to "%s": %s' % (url, ex)) 149 | 150 | 151 | def _instrument(): 152 | # monkeypatch SQLAlchemy __repr__ methods 153 | from . import formatter # noqa 154 | from . import loader # noqa 155 | 156 | 157 | def _set_output(path, overwrite=False): 158 | if os.path.exists(path) and not overwrite: 159 | print >>err, 'File "%s" exists and will be overwritten.' % path 160 | resp = raw_input('Overwrite (y/[n]): ') 161 | if not resp.strip().lower().startswith('y'): 162 | print >>err, "Aborted." 163 | sys.exit(-1) 164 | 165 | global out 166 | try: 167 | out = open(path, 'w') 168 | except IOError, e: 169 | print >>err, 'Could not open "%s" for writing: %s' % (path, e) 170 | sys.exit(-1) 171 | 172 | 173 | def configure(argv=sys.argv): 174 | global options, dburl 175 | 176 | parser = create_parser() 177 | options, args = parser.parse_args(argv) 178 | 179 | if len(args) < 2: 180 | parser.error("A database URL is required.") 181 | elif len(args) > 2: 182 | parser.error("Unknown arguments: %s" % (' '.join(args[2:]))) 183 | else: 184 | dburl = args[1] 185 | 186 | _version_check(parser) 187 | 188 | _setup_engine(parser, dburl) 189 | 190 | _instrument() 191 | 192 | if options.output is not None: 193 | _set_output(options.output, options.force) 194 | -------------------------------------------------------------------------------- /sqlautocode/constants.py: -------------------------------------------------------------------------------- 1 | # Nice print stuff 2 | TAB = 12 * ' ' 3 | 4 | NLTAB = ',\n' + TAB 5 | 6 | USAGE = """usage: autoload.py [options] 7 | 8 | Generates python code for a given database schema. 9 | 10 | options: 11 | -h, --help Show this help 12 | -u URL, --url URL Database url (e.g.: postgresql+psycopg2://postgres:user@password/Database) 13 | -o FILE, --output FILE Where to put the output (default is stdout) 14 | -s NAME, --schema NAME Name of the schema to output (default is 'default') 15 | -t T1,T2,.. , --tables T1,T2 Name of tables to inspect (default is 'all'). 16 | Support globbing character to select more tables. 17 | ex.: -t Download* will generate a model for all tables starting with Download 18 | 19 | -i --noindex Do not generate index information 20 | -g --generic-types Generate generic column types rather than database-specific type 21 | -e --example Generate code with examples how to access data 22 | -3 --z3c Generate code for use with z3c.sqlalchemy 23 | """ 24 | 25 | HEADER = """\ 26 | # -*- coding: %(encoding)s -*- 27 | ## File autogenerated by SQLAutoCode 28 | ## see http://code.google.com/p/sqlautocode/ 29 | 30 | from sqlalchemy import * 31 | %(dialect)s 32 | metadata = MetaData() 33 | """ 34 | 35 | HEADER_Z3C = """\ 36 | # -*- coding: %(encoding)s -*- 37 | ## File autogenerated by SQLAutoCode 38 | ## see http://code.google.com/p/sqlautocode/ 39 | ## Export type: z3c.sqlalchemy 40 | 41 | from sqlalchemy import * 42 | %(dialect)s 43 | from z3c.sqlalchemy import Model 44 | from z3c.sqlalchemy.mapper import MappedClassBase 45 | 46 | def getModel(metadata): 47 | model = Model() 48 | """ 49 | 50 | PG_IMPORT = """\ 51 | try: 52 | from sqlalchemy.dialects.postgresql import * 53 | except ImportError: 54 | from sqlalchemy.databases.postgres import * 55 | """ 56 | 57 | FOOTER_Z3C = """ 58 | return model 59 | """ 60 | 61 | FOOTER_EXAMPLE = """ 62 | # some example usage 63 | if __name__ == '__main__': 64 | db = create_engine(%(url)r) 65 | metadata.bind = db 66 | 67 | # fetch first 10 items from %(tablename)s 68 | s = %(tablename)s.select().limit(10) 69 | rs = s.execute() 70 | for row in rs: 71 | print row 72 | """ 73 | 74 | TABLE = """ Table('%(name)s', metadata, 75 | %(columns)s, 76 | %(constraints)s 77 | %(schema)s 78 | ) 79 | """ 80 | 81 | COLUMN = """Column(%(name)r, %(type)s%(constraints)s%(args)s)""" 82 | 83 | FOREIGN_KEY = """ForeignKeyConstraint(%(names)s, %(specs)s, name=%(name)s)""" 84 | 85 | INDEX = """Index(%(name)s, %(columns)s, unique=%(unique)s)""" 86 | 87 | HEADER_DECL = """#autogenerated by sqlautocode 88 | 89 | from sqlalchemy import * 90 | from sqlalchemy.ext.declarative import declarative_base 91 | from sqlalchemy.orm import relation 92 | 93 | engine = create_engine('%s') 94 | DeclarativeBase = declarative_base() 95 | metadata = DeclarativeBase.metadata 96 | metadata.bind = engine 97 | 98 | """ 99 | 100 | EXAMPLE_DECL = """#example on how to query your Schema 101 | from sqlalchemy.orm import sessionmaker 102 | session = sessionmaker(bind=engine)() 103 | objs = session.query(%s).all() 104 | print 'All %s objects: %%s'%%objs 105 | """ 106 | 107 | INTERACTIVE = """ 108 | print 'Trying to start IPython shell...', 109 | try: 110 | from IPython.Shell import IPShellEmbed 111 | print 'Success! Press to exit.' 112 | print 'Available models:%%s'%%%s 113 | print '\\nTry something like: session.query(%s).all()' 114 | ipshell = IPShellEmbed() 115 | ipshell() 116 | except: 117 | 'Failed. please easy_install ipython' 118 | """ 119 | -------------------------------------------------------------------------------- /sqlautocode/declarative.py: -------------------------------------------------------------------------------- 1 | import inspect 2 | import logging 3 | from util import name2label, plural, singular 4 | try: 5 | from cStringIO import StringIO 6 | except ImportError: 7 | from StringIO import StringIO 8 | 9 | import sqlalchemy 10 | from sqlalchemy import exc 11 | from sqlalchemy import MetaData, ForeignKeyConstraint 12 | from sqlalchemy.ext.declarative import declarative_base 13 | try: 14 | from sqlalchemy.ext.declarative import _deferred_relationship 15 | except ImportError: 16 | # SA 0.5 support 17 | try: 18 | from sqlalchemy.ext.declarative import _deferred_relation as _deferred_relationship 19 | except ImportError: 20 | # SA 0.8 support 21 | from sqlalchemy.ext.declarative.clsregistry import _deferred_relationship 22 | 23 | from sqlalchemy.orm import relation, class_mapper, Mapper 24 | 25 | try: 26 | # SA 0.5 support 27 | from sqlalchemy.orm import RelationProperty 28 | except ImportError: 29 | # SA 0.7 support 30 | try: 31 | from sqlalchemy.orm.properties import RelationshipProperty, RelationProperty 32 | except ImportError: 33 | RelationProperty = None 34 | 35 | from . import config 36 | from . import constants 37 | from .formatter import _repr_coltype_as 38 | 39 | 40 | log = logging.getLogger('saac.decl') 41 | log.setLevel(logging.DEBUG) 42 | handler = logging.StreamHandler() 43 | formatter = logging.Formatter("%(name)s - %(levelname)s - %(message)s") 44 | handler.setFormatter(formatter) 45 | log.addHandler(handler) 46 | 47 | 48 | def by_name(a, b): 49 | if a.name > b.name: 50 | return 1 51 | return -1 52 | 53 | 54 | def by__name__(a, b): # noqa 55 | if a.__name__ > b.__name__: 56 | return 1 57 | return -1 58 | 59 | 60 | def column_repr(self): 61 | 62 | kwarg = [] 63 | if self.key != self.name: 64 | kwarg.append('key') 65 | 66 | if hasattr(self, 'primary_key') and self.primary_key: 67 | self.primary_key = True 68 | kwarg.append('primary_key') 69 | 70 | # Kamil edit so we don't get errors when adding objects 71 | # without columns that have server defaults 72 | # see: http://docs.sqlalchemy.org/en/rel_0_8/core/schema.html#column-table-metadata-api 73 | if self.server_default: 74 | kwarg.append('server_default') 75 | setattr(self, 'server_default', "true") 76 | 77 | if not self.nullable: 78 | kwarg.append('nullable') 79 | if self.onupdate: 80 | kwarg.append('onupdate') 81 | if self.default: 82 | kwarg.append('default') 83 | ks = ', '.join('%s=%r' % (k, getattr(self, k)) for k in kwarg) 84 | 85 | if not hasattr(config, 'options') and self.config.options.generictypes: 86 | coltype = repr(self.type) 87 | elif type(self.type).__module__ == 'sqlalchemy.types': 88 | coltype = repr(self.type) 89 | else: 90 | # Try to 'cast' this column type to a cross-platform type 91 | # from sqlalchemy.types, dropping any database-specific type 92 | # arguments. 93 | for base in type(self.type).__mro__: 94 | if all((base.__module__ == 'sqlalchemy.types', 95 | base.__name__ in sqlalchemy.__all__)): 96 | coltype = _repr_coltype_as(self.type, base) 97 | break 98 | # FIXME: if a dialect has a non-standard type that does not 99 | # derive from an ANSI type, there's no choice but to ignore 100 | # generic-types and output the exact type. However, import 101 | # headers have already been output and lack the required 102 | # dialect import. 103 | else: 104 | coltype = repr(self.type) 105 | 106 | data = {'name': self.name, 107 | 'type': coltype, 108 | 'constraints': ', '.join(["ForeignKey('%s')" % cn.target_fullname for cn in self.foreign_keys]), 109 | 'args': ks and ks or '', 110 | } 111 | 112 | if data['constraints']: 113 | if data['constraints']: 114 | data['constraints'] = ', ' + data['constraints'] 115 | if data['args']: 116 | if data['args']: 117 | data['args'] = ', ' + data['args'] 118 | 119 | return constants.COLUMN % data 120 | 121 | 122 | class ModelFactory(object): 123 | 124 | def __init__(self, config): 125 | self.config = config 126 | self.used_model_names = [] 127 | self.used_table_names = [] 128 | self.table_model_dict = {} # Kamil Edit 129 | schema = getattr(self.config, 'schema', None) 130 | self._metadata = MetaData(bind=config.engine) 131 | self._foreign_keys = {} 132 | self.schemas = None 133 | if schema: 134 | if isinstance(schema, (list, tuple)): 135 | self.schemas = schema 136 | else: 137 | self.schemas = (schema, ) 138 | for schema in self.schemas: 139 | log.info('Reflecting database... schema:%s' % schema) 140 | self._metadata.reflect(schema=schema) 141 | else: 142 | log.info('Reflecting database...') 143 | self._metadata.reflect() 144 | 145 | self.DeclarativeBase = declarative_base(metadata=self._metadata) 146 | 147 | def _table_repr(self, table): 148 | s = "Table(u'%s', metadata,\n" % (table.name) 149 | for column in table.c: 150 | s += " %s,\n" % column_repr(column) 151 | if table.schema: 152 | s += " schema='%s'\n" % table.schema 153 | s += ")" 154 | return s 155 | 156 | def __repr__(self): 157 | tables = self.get_many_to_many_tables() 158 | tables.extend(self.get_tables_with_no_pks()) 159 | models = self.models 160 | 161 | s = StringIO() 162 | engine = self.config.engine 163 | if not isinstance(engine, basestring): 164 | engine = str(engine.url) 165 | s.write(constants.HEADER_DECL % engine) 166 | if 'postgres' in engine: 167 | s.write(constants.PG_IMPORT) 168 | 169 | self.used_table_names = [] 170 | self.used_model_names = [] 171 | self.table_model_dict = {} # Kamil Edit 172 | for table in tables: 173 | if table not in self.tables: 174 | continue 175 | table_name = self.find_new_name(table.name, self.used_table_names) 176 | self.used_table_names.append(table_name) 177 | s.write('%s = %s\n\n' % (table_name, self._table_repr(table))) 178 | 179 | for model in models: 180 | s.write(model.__repr__()) 181 | s.write("\n\n") 182 | 183 | if self.config.example or self.config.interactive: 184 | s.write(constants.EXAMPLE_DECL % (models[0].__name__, models[0].__name__)) 185 | if self.config.interactive: 186 | s.write(constants.INTERACTIVE % ([model.__name__ for model in models], models[0].__name__)) 187 | return s.getvalue() 188 | 189 | @property 190 | def tables(self): 191 | if self.config.options.tables: 192 | tables = set(self.config.options.tables) 193 | return [self._metadata.tables[t] for t in set(self._metadata.tables.keys()).intersection(tables)] 194 | return self._metadata.tables.values() 195 | 196 | @property 197 | def table_names(self): 198 | return [t.name for t in self.tables] 199 | 200 | @property 201 | def models(self): 202 | if hasattr(self, '_models'): 203 | return self._models 204 | self.used_model_names = [] 205 | self.used_table_names = [] 206 | self.table_model_dict = {} # Kamil Edit 207 | self._models = [] 208 | for table in self.get_non_many_to_many_tables(): 209 | try: 210 | self._models.append(self.create_model(table)) 211 | except exc.ArgumentError: 212 | log.warning("Table with name %s ha no primary key. No ORM class created" % table.name) 213 | self._models.sort(by__name__) 214 | return self._models 215 | 216 | def get_tables_with_no_pks(self): 217 | r = [] 218 | for table in self.get_non_many_to_many_tables(): 219 | if not [c for c in table.columns if c.primary_key]: 220 | r.append(table) 221 | return r 222 | 223 | def model_table_lookup(self): 224 | if hasattr(self, '_model_table_lookup'): 225 | return self._model_table_lookup 226 | self._model_table_lookup = dict(((m.__table__.name, m.__name__) for m in self.models)) 227 | return self._model_table_lookup 228 | 229 | def find_new_name(self, prefix, used, i=0): 230 | if i != 0: 231 | prefix = "%s%d" % (prefix, i) 232 | if prefix in used: 233 | prefix = prefix 234 | return self.find_new_name(prefix, used, i + 1) 235 | return prefix 236 | 237 | def create_model(self, table): 238 | # partially borrowed from Jorge Vargas' code 239 | # http://dpaste.org/V6YS/ 240 | log.debug('Creating Model from table: %s' % table.name) 241 | 242 | model_name = self.find_new_name(singular(name2label(table.name)), self.used_model_names) 243 | self.used_model_names.append(model_name) 244 | is_many_to_many_table = self.is_many_to_many_table(table) 245 | table_name = self.find_new_name(table.name, self.used_table_names) 246 | self.used_table_names.append(table_name) 247 | self.table_model_dict[table_name] = model_name # Kamil Edit 248 | 249 | mtl = self.model_table_lookup 250 | 251 | class Temporal(self.DeclarativeBase): 252 | __table__ = table 253 | 254 | @classmethod 255 | def _relation_repr(cls, rel): 256 | target = rel.argument 257 | if target and inspect.isfunction(target): 258 | target = target() 259 | if isinstance(target, Mapper): 260 | target = target.class_ 261 | target = target.__name__ 262 | primaryjoin = '' 263 | lookup = mtl() 264 | foo = rel.key 265 | if rel.primaryjoin is not None and hasattr(rel.primaryjoin, 'right'): 266 | right_lookup = lookup.get(rel.primaryjoin.right.table.name, '%s.c' % rel.primaryjoin.right.table.name) 267 | left_lookup = lookup.get(rel.primaryjoin.left.table.name, '%s.c' % rel.primaryjoin.left.table.name) 268 | 269 | primaryjoin = ", primaryjoin='%s.%s==%s.%s'" % (left_lookup, 270 | rel.primaryjoin.left.name, 271 | right_lookup, 272 | rel.primaryjoin.right.name) 273 | elif hasattr(rel, '_as_string'): 274 | primaryjoin = ', primaryjoin="%s"' % rel._as_string 275 | 276 | secondary = '' 277 | secondaryjoin = '' 278 | if rel.secondary is not None: 279 | """ 280 | **HACK**: If there is a secondary relationship like between Venue, Event, and Event_Type, then I'm only 281 | going show a primary relationship. 282 | "Events = relationship('Event', primaryjoin='Venue.id==Event.venue_id')" and not 283 | "Event_Types = relation('EventType', primaryjoin='Venue.id==Event.venue_id', secondary=Event, secondaryjoin='Event.event_type_id==EventType.id')" 284 | """ 285 | if rel.secondary.name in self.table_model_dict: 286 | target = self.table_model_dict[rel.secondary.name] 287 | else: 288 | target = self.find_new_name(singular(name2label(rel.secondary.name)), []) # **HACK** 289 | secondary = '' 290 | secondaryjoin = '' 291 | foo = plural(rel.secondary.name) 292 | backref = '' 293 | # if rel.backref: 294 | # backref=", backref='%s'"%rel.backref.key 295 | return "%s = relationship('%s'%s%s%s%s)" % (foo, target, primaryjoin, secondary, secondaryjoin, backref) 296 | 297 | @classmethod 298 | def __repr__(cls): 299 | log.debug('repring class with name %s' % cls.__name__) 300 | try: 301 | mapper = None 302 | try: 303 | mapper = class_mapper(cls) 304 | except exc.InvalidRequestError: 305 | log.warn("A proper mapper could not be generated for the class %s, no relations will be created" % model_name) 306 | s = "" 307 | s += "class " + model_name + '(DeclarativeBase):\n' 308 | if is_many_to_many_table: 309 | s += " __table__ = %s\n\n" % table_name 310 | else: 311 | s += " __tablename__ = '%s'\n\n" % table_name 312 | if hasattr(cls, '__table_args__'): 313 | # if cls.__table_args__[0]: 314 | # for fkc in cls.__table_args__[0]: 315 | # fkc.__class__.__repr__ = foreignkeyconstraint_repr 316 | # break 317 | s += " __table_args__ = %s\n\n" % cls.__table_args__ 318 | s += " #column definitions\n" 319 | for column in sorted(cls.__table__.c, by_name): 320 | s += " %s = %s\n" % (column.name, column_repr(column)) 321 | s += "\n #relation definitions\n" 322 | # this is only required in SA 0.5 323 | if mapper and RelationProperty: 324 | for prop in mapper.iterate_properties: 325 | if isinstance(prop, RelationshipProperty): 326 | s += ' %s\n' % cls._relation_repr(prop) 327 | return s 328 | 329 | except Exception: 330 | log.error("Could not generate class for: %s" % cls.__name__) 331 | from traceback import format_exc 332 | log.error(format_exc()) 333 | return '' 334 | 335 | # hack the class to have the right classname 336 | Temporal.__name__ = model_name 337 | 338 | # set up some blank table args 339 | Temporal.__table_args__ = {} 340 | 341 | # add in the schema 342 | if self.config.schema: 343 | Temporal.__table_args__[1]['schema'] = table.schema 344 | 345 | # trick sa's model registry to think the model is the correct name 346 | if model_name != 'Temporal': 347 | Temporal._decl_class_registry[model_name] = Temporal._decl_class_registry['Temporal'] 348 | del Temporal._decl_class_registry['Temporal'] 349 | 350 | # add in single relations 351 | fks = self.get_single_foreign_keys_by_column(table) 352 | for column, fk in fks.iteritems(): 353 | related_table = fk.column.table 354 | if related_table not in self.tables: 355 | continue 356 | 357 | log.info(' Adding foreign key for:%s' % related_table.name) 358 | rel = relation(singular(name2label(related_table.name, related_table.schema)), 359 | primaryjoin=column == fk.column) # , backref=backref_name) 360 | 361 | setattr(Temporal, related_table.name, _deferred_relationship(Temporal, rel)) 362 | 363 | # add in the relations for the composites 364 | for constraint in table.constraints: 365 | if isinstance(constraint, ForeignKeyConstraint): 366 | if len(constraint.elements) > 1: 367 | related_table = constraint.elements[0].column.table 368 | related_classname = singular(name2label(related_table.name, related_table.schema)) 369 | 370 | primary_join = "and_(%s)" % ', '.join(["%s.%s==%s.%s" % ( 371 | model_name, 372 | k.parent.name, 373 | related_classname, 374 | k.column.name 375 | ) for k in constraint.elements]) 376 | rel = relation( 377 | related_classname, 378 | primaryjoin=primary_join 379 | # foreign_keys=[k.parent for k in constraint.elements] 380 | ) 381 | 382 | rel._as_string = primary_join 383 | setattr(Temporal, related_table.name, rel) # _deferred_relationship(Temporal, rel)) 384 | 385 | # add in many-to-many relations 386 | for join_table in self.get_related_many_to_many_tables(table.name): 387 | 388 | if join_table not in self.tables: 389 | continue 390 | primary_column = [c for c in join_table.columns if c.foreign_keys and list(c.foreign_keys)[0].column.table == table][0] 391 | 392 | for column in join_table.columns: 393 | if column.foreign_keys: 394 | key = list(column.foreign_keys)[0] 395 | if key.column.table is not table: 396 | related_column = related_table = list(column.foreign_keys)[0].column 397 | related_table = related_column.table 398 | if related_table not in self.tables: 399 | continue 400 | log.info(' Adding foreign key(%s) for:%s' % (key, related_table.name)) 401 | setattr( 402 | Temporal, 403 | plural(related_table.name), 404 | _deferred_relationship( 405 | Temporal, 406 | relation( 407 | singular(name2label(related_table.name, related_table.schema)), 408 | secondary=join_table, 409 | primaryjoin=list(primary_column.foreign_keys)[0].column == primary_column, 410 | secondaryjoin=column == related_column 411 | ) 412 | ) 413 | ) 414 | break 415 | 416 | return Temporal 417 | 418 | def get_table(self, name): 419 | """(name) -> sqlalchemy.schema.Table 420 | get the table definition with the given table name 421 | """ 422 | if self.schemas: 423 | for schema in self.schemas: 424 | if schema and not name.startswith(schema): 425 | new_name = '.'.join((schema, name)) 426 | table = self._metadata.tables.get(new_name, None) 427 | if table is not None: 428 | return table 429 | return self._metadata.tables[name] 430 | 431 | def get_single_foreign_keys_by_column(self, table): 432 | keys_by_column = {} 433 | fks = self.get_foreign_keys(table) 434 | for table, keys in fks.iteritems(): 435 | if len(keys) == 1: 436 | fk = keys[0] 437 | keys_by_column[fk.parent] = fk 438 | return keys_by_column 439 | 440 | def get_composite_foreign_keys(self, table): 441 | l = [] 442 | fks = self.get_foreign_keys(table) 443 | for table, keys in fks.iteritems(): 444 | if len(keys) > 1: 445 | l.append(keys) 446 | return l 447 | 448 | def get_foreign_keys(self, table): 449 | if table in self._foreign_keys: 450 | return self._foreign_keys[table] 451 | 452 | fks = table.foreign_keys 453 | 454 | # group fks by table. I think this is needed because of a problem in the sa reflection alg. 455 | grouped_fks = {} 456 | for key in fks: 457 | grouped_fks.setdefault(key.column.table, []).append(key) 458 | 459 | self._foreign_keys[table] = grouped_fks 460 | return grouped_fks 461 | 462 | # fks = {} 463 | # for column in table.columns: 464 | # if len(column.foreign_keys)>0: 465 | # fks.setdefault(column.name, []).extend(column.foreign_keys) 466 | # return fks 467 | 468 | def is_many_to_many_table(self, table): 469 | fks = self.get_single_foreign_keys_by_column(table).values() 470 | return len(fks) >= 2 471 | 472 | def is_only_many_to_many_table(self, table): 473 | return len(self.get_single_foreign_keys_by_column(table)) == 2 and len(table.c) == 2 474 | 475 | def get_many_to_many_tables(self): 476 | if not hasattr(self, '_many_to_many_tables'): 477 | self._many_to_many_tables = [table for table in self._metadata.tables.values() if self.is_many_to_many_table(table)] 478 | return sorted(self._many_to_many_tables, by_name) 479 | 480 | def get_non_many_to_many_tables(self): 481 | tables = [table for table in self.tables if not(self.is_only_many_to_many_table(table))] 482 | return sorted(tables, by_name) 483 | 484 | def get_related_many_to_many_tables(self, table_name): 485 | tables = [] 486 | src_table = self.get_table(table_name) 487 | for table in self.get_many_to_many_tables(): 488 | for column in table.columns: 489 | if column.foreign_keys: 490 | key = list(column.foreign_keys)[0] 491 | if key.column.table is src_table: 492 | tables.append(table) 493 | break 494 | return sorted(tables, by_name) 495 | -------------------------------------------------------------------------------- /sqlautocode/formatter.py: -------------------------------------------------------------------------------- 1 | import sqlalchemy 2 | 3 | from . import config, constants, util 4 | 5 | 6 | def textclause_repr(self): 7 | return 'text(%r)' % self.text 8 | 9 | 10 | def table_repr(self): 11 | data = { 12 | 'name': self.name, 13 | 'columns': constants.NLTAB.join([repr(cl) for cl in self.columns]), 14 | 'constraints': constants.NLTAB.join( 15 | [repr(cn) for cn in self.constraints 16 | if not isinstance(cn, sqlalchemy.PrimaryKeyConstraint)]), 17 | 'index': '', 18 | 'schema': self.schema is not None and "schema='%s'" % self.schema or '', 19 | } 20 | 21 | if data['constraints']: 22 | data['constraints'] = data['constraints'] + ',' 23 | 24 | return util.as_out_str(constants.TABLE % data) 25 | 26 | 27 | def _repr_coltype_as(coltype, as_type): 28 | """repr a Type instance as a super type.""" 29 | 30 | specimen = object.__new__(as_type) 31 | specimen.__dict__ = coltype.__dict__ 32 | return repr(specimen) 33 | 34 | 35 | def column_repr(self): 36 | kwarg = [] 37 | if self.key != self.name: 38 | kwarg.append('key') 39 | 40 | if hasattr(self, 'primary_key'): 41 | kwarg.append('primary_key') 42 | 43 | if not self.nullable: 44 | kwarg.append('nullable') 45 | if self.onupdate: 46 | kwarg.append('onupdate') 47 | if self.default: 48 | kwarg.append('default') 49 | elif self.server_default: 50 | self.default = self.server_default.arg 51 | kwarg.append('default') 52 | 53 | ks = ', '.join('%s=%r' % (k, getattr(self, k)) for k in kwarg) 54 | 55 | if not hasattr(config, 'options') and config.options.generictypes: 56 | coltype = repr(self.type) 57 | elif type(self.type).__module__ == 'sqlalchemy.types': 58 | coltype = repr(self.type) 59 | else: 60 | # Try to 'cast' this column type to a cross-platform type 61 | # from sqlalchemy.types, dropping any database-specific type 62 | # arguments. 63 | for base in type(self.type).__mro__: 64 | if all((base.__module__ == 'sqlalchemy.types', 65 | base.__name__ in sqlalchemy.__all__)): 66 | coltype = _repr_coltype_as(self.type, base) 67 | break 68 | # FIXME: if a dialect has a non-standard type that does not 69 | # derive from an ANSI type, there's no choice but to ignore 70 | # generic-types and output the exact type. However, import 71 | # headers have already been output and lack the required 72 | # dialect import. 73 | else: 74 | coltype = repr(self.type) 75 | 76 | data = {'name': self.name, 77 | 'type': coltype, 78 | 'constraints': ', '.join([repr(cn) for cn in self.constraints]), 79 | 'args': ks and ks or '', 80 | } 81 | 82 | if data['constraints']: 83 | data['constraints'] = ', ' + data['constraints'] 84 | if data['args']: 85 | data['args'] = ', ' + data['args'] 86 | 87 | return util.as_out_str(constants.COLUMN % data) 88 | 89 | 90 | def foreignkeyconstraint_repr(self): 91 | data = {'name': repr(self.name), 92 | 'names': repr([x.parent.name for x in self.elements]), 93 | 'specs': repr([x._get_colspec() for x in self.elements]) 94 | } 95 | return util.as_out_str(constants.FOREIGN_KEY % data) 96 | 97 | 98 | def index_repr(index): 99 | cols = [] 100 | for column in index.columns: 101 | # FIXME: still punting on the issue of unicode table names 102 | if util.is_python_identifier(column.name): 103 | cols.append('%s.c.%s' % (column.table.name, column.name)) 104 | else: 105 | cols.append('%s.c[%r]' % (column.table.name, column.name)) 106 | 107 | data = {'name': repr(index.name), 108 | 'columns': ', '.join(cols), 109 | 'unique': repr(index.unique), 110 | } 111 | return util.as_out_str(constants.INDEX % data) 112 | 113 | 114 | def monkey_patch_sa(): 115 | sqlalchemy.sql.expression._TextClause.__repr__ = textclause_repr 116 | sqlalchemy.schema.Table.__repr__ = table_repr 117 | sqlalchemy.schema.Column.__repr__ = column_repr 118 | sqlalchemy.schema.ForeignKeyConstraint.__repr__ = foreignkeyconstraint_repr 119 | sqlalchemy.schema.Index.__repr__ = index_repr 120 | -------------------------------------------------------------------------------- /sqlautocode/loader.py: -------------------------------------------------------------------------------- 1 | import sqlalchemy 2 | from sqlalchemy.databases import postgres 3 | 4 | 5 | class AutoLoader(object): 6 | pass 7 | 8 | 9 | class PGIndexLoader(AutoLoader): 10 | """ SA does not load indexes for us """ 11 | 12 | sql4indexes = "SELECT indexname, tablename, indexdef FROM pg_indexes" 13 | 14 | def __init__(self, db): 15 | ix = {} 16 | for name, tbl_name, sqltext in db.execute(self.sql4indexes): 17 | ix.setdefault(tbl_name, []).append((name, sqltext)) 18 | self._indexes = ix 19 | 20 | def indexes(self, table): 21 | return [self._index_from_def(name, sqltext, table) 22 | for name, sqltext in self._indexes.get(table.name, ())] 23 | 24 | def _index_from_def(self, name, sqltext, table): 25 | # CREATE UNIQUE INDEX name ON "tablename" USING btree (columnslist) 26 | unique = ' UNIQUE ' in sqltext 27 | cols = sqltext.split(' (')[1].split(')')[0].split(',') 28 | cols = [table.columns[cname.strip().replace('"', '')] 29 | for cname in cols] 30 | name = name.encode('utf-8') 31 | return sqlalchemy.Index(name, unique=unique, *cols) 32 | 33 | postgres.PGDialect.indexloader = PGIndexLoader 34 | -------------------------------------------------------------------------------- /sqlautocode/main.py: -------------------------------------------------------------------------------- 1 | import sys 2 | 3 | from . import config 4 | from . import constants 5 | from . import util 6 | from .declarative import ModelFactory 7 | from .util import emit 8 | 9 | 10 | def main(): 11 | config.configure() 12 | 13 | options = config.options 14 | if options.declarative: 15 | config.interactive = None 16 | if options.interactive: 17 | config.interactive = True 18 | config.schema = None 19 | if options.schema: 20 | config.schema = options.schema 21 | config.example = False 22 | if options.example: 23 | config.example = True 24 | factory = ModelFactory(config) 25 | emit(repr(factory)) 26 | config.out.close() 27 | config.out = sys.stdout 28 | print >>config.err, "Output written to %s" % options.output 29 | return 30 | 31 | import formatter 32 | formatter.monkey_patch_sa() 33 | 34 | import sqlalchemy 35 | from sqlalchemy.engine.reflection import Inspector 36 | db, options = config.engine, config.options 37 | metadata = sqlalchemy.MetaData(db) 38 | 39 | print >>config.err, 'Starting...' 40 | conn = db.connect() 41 | inspector = Inspector.from_engine(conn) 42 | 43 | if options.schema is not None: 44 | reflection_schema = options.schema 45 | else: 46 | try: 47 | reflection_schema = inspector.default_schema_name 48 | except NotImplementedError: 49 | reflection_schema = None 50 | 51 | tablenames = ( 52 | inspector.get_table_names(reflection_schema) 53 | + inspector.get_view_names(reflection_schema) 54 | ) 55 | 56 | # fixme: don't set up output until we're sure there's work to do! 57 | if options.tables: 58 | subset, missing, unglobbed = util.glob_intersection(tablenames, 59 | options.tables) 60 | for identifier in missing: 61 | print >>config.err, 'Table "%s" not found.' % identifier 62 | for glob in unglobbed: 63 | print >>config.err, '"%s" matched no tables.' % glob 64 | if not subset: 65 | print >>config.err, "No tables matched!" 66 | sys.exit(1) 67 | 68 | tablenames = subset 69 | 70 | # some header with imports 71 | if options.generictypes: 72 | dialect = '' 73 | else: 74 | dialect = 'from sqlalchemy.databases.%s import *\n' % db.name 75 | 76 | header = options.z3c and constants.HEADER_Z3C or constants.HEADER 77 | emit(header % {'dialect': dialect, 'encoding': options.encoding}) 78 | 79 | for tname in tablenames: 80 | print >>config.err, "Generating python model for table %s" % ( 81 | util.as_sys_str(tname)) 82 | 83 | table = sqlalchemy.Table(tname, metadata, schema=reflection_schema, 84 | autoload=True) 85 | if options.schema is None: 86 | # we're going to remove the schema from the table so that it 87 | # isn't rendered in the output. If we don't put back the 88 | # correct value, it may cause errors when other tables reference 89 | # this one. 90 | original_schema = table.schema 91 | table.schema = None 92 | else: 93 | original_schema = options.schema 94 | 95 | inc = '\n\n' 96 | if options.z3c: 97 | inc = inc + 4 * ' ' 98 | 99 | emit('%s%s%s%s = %r' % (inc, 100 | options.table_prefix, 101 | tname, 102 | options.table_suffix, table)) 103 | 104 | if options.z3c: 105 | emit(inc + ('class %(tn)sObject(MappedClassBase): pass\n' 106 | 'mapper(%(tn)sObject, %(tn)s)') % {'tn': tname}) 107 | 108 | table.schema = original_schema 109 | 110 | # directly print indices after table def 111 | if not options.noindex: 112 | indexes = [] 113 | if not table.indexes: 114 | # for certain dialects we need to include index support 115 | if hasattr(db.dialect, 'indexloader'): 116 | indexes = db.dialect.indexloader(db).indexes(table) 117 | else: 118 | print >> config.err, ( 119 | 'It seems that this dialect does not support indexes!' 120 | ) 121 | else: 122 | indexes = list(table.indexes) 123 | 124 | util.emit(*[repr(index) for index in indexes]) 125 | 126 | if options.z3c: 127 | emit(constants.FOOTER_Z3C) 128 | 129 | # print some example 130 | if options.example: 131 | emit('\n' + constants.FOOTER_EXAMPLE % { 132 | 'url': unicode(db.url), 'tablename': tablenames[0]}) 133 | 134 | if options.output: 135 | emit('\n') 136 | config.out.close() 137 | config.out = sys.stdout 138 | print >> config.err, "Output written to %s" % options.output 139 | 140 | # vim:ts=4:sw=4:expandtab 141 | -------------------------------------------------------------------------------- /sqlautocode/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ksindi/sqlautocode/5e9eaf4c70ede361de3397e4346165fa403ca9c0/sqlautocode/tests/__init__.py -------------------------------------------------------------------------------- /sqlautocode/tests/base.py: -------------------------------------------------------------------------------- 1 | import os 2 | from sqlalchemy import * 3 | 4 | 5 | metadata = MetaData() 6 | 7 | environment = Table('environment', metadata, 8 | Column(u'environment_id', Numeric(precision=10, scale=0, asdecimal=True), primary_key=True, nullable=False), 9 | Column(u'environment_name', String(length=100, convert_unicode=False, assert_unicode=None), primary_key=False, nullable=False), 10 | Column(u'database_host', String(length=100, convert_unicode=False, assert_unicode=None), primary_key=False, nullable=False), 11 | Column(u'database_port', String(length=5, convert_unicode=False, assert_unicode=None), primary_key=False, nullable=False), 12 | Column(u'database_sid', String(length=32, convert_unicode=False, assert_unicode=None), primary_key=False, nullable=False), 13 | Column(u'database_user', String(length=100, convert_unicode=False, assert_unicode=None), primary_key=False, nullable=False), 14 | Column(u'database_pass', String(length=100, convert_unicode=False, assert_unicode=None), primary_key=False, nullable=False), 15 | 16 | 17 | ) 18 | 19 | 20 | report = Table('report', metadata, 21 | Column(u'report_id', Numeric(precision=10, scale=0, asdecimal=True), primary_key=True, nullable=False), 22 | Column(u'environment_id', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False, nullable=False), 23 | Column(u'report_name', String(length=50, convert_unicode=False, assert_unicode=None), primary_key=False, nullable=False), 24 | Column(u'report_description', String(length=4000, convert_unicode=False, assert_unicode=None), primary_key=False), 25 | Column(u'deleted', Numeric(precision=1, scale=0, asdecimal=True), primary_key=False, nullable=False), 26 | Column(u'created_date', DateTime(timezone=False), primary_key=False, nullable=False), 27 | Column(u'created_by', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False, nullable=False), 28 | Column(u'updated_date', DateTime(timezone=False), primary_key=False, nullable=False), 29 | Column(u'updated_by', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False, nullable=False), 30 | Column(u'deleted_date', DateTime(timezone=False), primary_key=False), 31 | Column(u'deleted_by', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False), 32 | ForeignKeyConstraint([u'environment_id'], [u'environment.environment_id'], name='REPORT_FK_ENV_ID'), 33 | 34 | ) 35 | 36 | 37 | ui_report = Table('ui_report', metadata, 38 | Column(u'ui_report_id', Numeric(precision=10, scale=0, asdecimal=True), primary_key=True, nullable=False), 39 | Column(u'report_id', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False, nullable=False), 40 | Column(u'environment_id', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False, nullable=False), 41 | Column(u'ui_report_name', String(length=100, convert_unicode=False, assert_unicode=None), primary_key=False, nullable=False), 42 | Column(u'ui_report_description', String(length=4000, convert_unicode=False, assert_unicode=None), primary_key=False), 43 | Column(u'enabled', Numeric(precision=1, scale=0, asdecimal=True), primary_key=False, nullable=False), 44 | Column(u'deleted', Numeric(precision=1, scale=0, asdecimal=True), primary_key=False, nullable=False), 45 | Column(u'created_date', DateTime(timezone=False), primary_key=False, nullable=False), 46 | Column(u'created_by', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False, nullable=False), 47 | Column(u'updated_date', DateTime(timezone=False), primary_key=False, nullable=False), 48 | Column(u'updated_by', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False, nullable=False), 49 | Column(u'deleted_date', DateTime(timezone=False), primary_key=False), 50 | Column(u'deleted_by', Numeric(precision=10, scale=0, asdecimal=True), primary_key=False), 51 | ForeignKeyConstraint([u'report_id'], [u'report.report_id'], name='UI_REPORT_FK_REPORT_ID'), 52 | ForeignKeyConstraint([u'environment_id'], [u'environment.environment_id'], name='UI_REPORT_FK_ENV_ID'), 53 | 54 | ) 55 | 56 | bound = False 57 | def make_test_db(): 58 | global bound, metadata 59 | if not bound: 60 | testdb_filename = os.path.abspath(os.path.dirname(__file__))+'/data/testdb.db' 61 | #try: 62 | # os.remove(testdb_filename) 63 | #except OSError: 64 | # pass 65 | 66 | db = 'sqlite:///'+testdb_filename 67 | 68 | test_engine = create_engine(db) 69 | metadata.bind =test_engine 70 | #metadata.create_all() 71 | bound = True 72 | return metadata 73 | 74 | bound_multi = False 75 | metadata_multi = MetaData() 76 | def make_test_db_multi(): 77 | global bound_multi, metadata_multi 78 | if not bound_multi: 79 | testdb_filename = os.path.abspath(os.path.dirname(__file__))+'/data/multi.db' 80 | #testdb_filename = os.path.abspath(os.path.dirname(__file__))+'/data/devdata.db' 81 | 82 | db = 'sqlite:///'+testdb_filename 83 | 84 | test_engine = create_engine(db) 85 | metadata_multi.bind =test_engine 86 | metadata_multi.reflect() 87 | bound_multi = True 88 | return metadata_multi 89 | 90 | 91 | -------------------------------------------------------------------------------- /sqlautocode/tests/test_declarative.py: -------------------------------------------------------------------------------- 1 | import os 2 | import pprint as pp 3 | from nose.tools import eq_ 4 | from sqlautocode.declarative import ModelFactory 5 | from sqlalchemy.orm import class_mapper 6 | testdb = 'sqlite:///'+os.path.abspath(os.path.dirname(__file__))+'/data/devdata.db' 7 | #testdb = 'postgres://postgres@localhost/TestSamples' 8 | 9 | from base import make_test_db, make_test_db_multi 10 | 11 | class DummyOptions: 12 | 13 | tables = None 14 | 15 | class DummyConfig: 16 | 17 | 18 | def __init__(self, engine=testdb): 19 | self.engine = engine 20 | 21 | example = True 22 | schema = None 23 | interactive = None 24 | 25 | options=DummyOptions() 26 | # schema = ['pdil_samples', 'pdil_tools'] 27 | 28 | 29 | class _TestModelFactory: 30 | 31 | def setup(self): 32 | self.config = DummyConfig() 33 | self.factory = ModelFactory(self.config) 34 | 35 | def test_tables(self): 36 | tables = sorted([t.name for t in self.factory.tables]) 37 | eq_(tables, [u'tg_group', u'tg_group_permission', u'tg_permission', u'tg_town', u'tg_user', u'tg_user_group']) 38 | 39 | def _setup_all_models(self): 40 | return self.factory.models 41 | 42 | def test_create_model(self): 43 | self.factory.used_model_names = [] 44 | self.factory.used_table_names = [] 45 | 46 | Group = self.factory.create_model(self.factory._metadata.tables['tg_group']) 47 | Permission = self.factory.create_model(self.factory._metadata.tables['tg_permission']) 48 | Town = self.factory.create_model(self.factory._metadata.tables['tg_town']) 49 | User = self.factory.create_model(self.factory._metadata.tables['tg_user']) 50 | 51 | class_mapper(Town) 52 | class_mapper(User) 53 | 54 | t = Town(town_name="Arvada") 55 | u = User(tg_town=t) 56 | assert u.tg_town.town_name == 'Arvada' 57 | 58 | def test_get_many_to_many_tables(self): 59 | tables = sorted([table.name for table in self.factory.get_many_to_many_tables()]) 60 | eq_(tables, [u'tg_group_permission', u'tg_user_group']) 61 | 62 | def test_get_related_many_to_many_tables(self): 63 | tables = [table.name for table in self.factory.get_related_many_to_many_tables('tg_user')] 64 | eq_(tables, [u'tg_user_group']) 65 | 66 | def test_get_foreign_keys(self): 67 | columns = [c[0].column.name for c in self.factory.get_foreign_keys(self.factory._metadata.tables['tg_user']).values()] 68 | eq_(columns, ['town_id']) 69 | 70 | def test_model___repr__(self): 71 | models = sorted(self._setup_all_models()) 72 | for model in models: 73 | if model.__name__=='TgUser': 74 | User = model 75 | r = User.__repr__() 76 | print r 77 | expected = """\ 78 | class TgUser(DeclarativeBase): 79 | __tablename__ = 'tg_user' 80 | 81 | #column definitions 82 | created = Column(u'created', TIMESTAMP(timezone=False)) 83 | display_name = Column(u'display_name', VARCHAR(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 84 | email_address = Column(u'email_address', VARCHAR(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 85 | password = Column(u'password', VARCHAR(length=80, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 86 | town_id = Column(u'town_id', INTEGER(), ForeignKey('tg_town.town_id')) 87 | user_id = Column(u'user_id', INTEGER(), primary_key=True, nullable=False) 88 | user_name = Column(u'user_name', VARCHAR(length=16, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 89 | 90 | #relation definitions 91 | tg_town = relation('TgTown', primaryjoin='TgUser.town_id==TgTown.town_id') 92 | tg_groups = relation('TgGroup', primaryjoin='TgUser.user_id==tg_user_group.c.user_id', secondary=tg_user_group, secondaryjoin='tg_user_group.c.group_id==TgGroup.group_id') 93 | """ 94 | eq_(r.strip(), expected.strip()) 95 | 96 | def test__repr__(self): 97 | 98 | r = repr(self.factory) 99 | print r 100 | expected = """\ 101 | #autogenerated by sqlautocode 102 | 103 | from sqlalchemy import * 104 | from sqlalchemy.ext.declarative import declarative_base 105 | from sqlalchemy.orm import relation 106 | 107 | engine = create_engine('sqlite:////Users/percious/oss/tgdev-py27/src/sqlautocode/sqlautocode/tests/data/devdata.db') 108 | DeclarativeBase = declarative_base() 109 | metadata = DeclarativeBase.metadata 110 | metadata.bind = engine 111 | 112 | tg_group_permission = Table(u'tg_group_permission', metadata, 113 | Column(u'group_id', INTEGER(), ForeignKey('tg_group.group_id'), primary_key=True, nullable=False), 114 | Column(u'permission_id', INTEGER(), ForeignKey('tg_permission.permission_id'), primary_key=True, nullable=False), 115 | ) 116 | 117 | tg_user_group = Table(u'tg_user_group', metadata, 118 | Column(u'user_id', INTEGER(), ForeignKey('tg_user.user_id'), primary_key=True, nullable=False), 119 | Column(u'group_id', INTEGER(), ForeignKey('tg_group.group_id'), primary_key=True, nullable=False), 120 | ) 121 | 122 | class TgGroup(DeclarativeBase): 123 | __tablename__ = 'tg_group' 124 | 125 | #column definitions 126 | created = Column(u'created', TIMESTAMP(timezone=False)) 127 | display_name = Column(u'display_name', VARCHAR(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 128 | group_id = Column(u'group_id', INTEGER(), primary_key=True, nullable=False) 129 | group_name = Column(u'group_name', VARCHAR(length=16, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 130 | 131 | #relation definitions 132 | tg_permissions = relation('TgPermission', primaryjoin='TgGroup.group_id==tg_group_permission.c.group_id', secondary=tg_group_permission, secondaryjoin='tg_group_permission.c.permission_id==TgPermission.permission_id') 133 | tg_users = relation('TgUser', primaryjoin='TgGroup.group_id==tg_user_group.c.group_id', secondary=tg_user_group, secondaryjoin='tg_user_group.c.user_id==TgUser.user_id') 134 | 135 | 136 | class TgPermission(DeclarativeBase): 137 | __tablename__ = 'tg_permission' 138 | 139 | #column definitions 140 | description = Column(u'description', VARCHAR(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 141 | permission_id = Column(u'permission_id', INTEGER(), primary_key=True, nullable=False) 142 | permission_name = Column(u'permission_name', VARCHAR(length=16, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 143 | 144 | #relation definitions 145 | tg_groups = relation('TgGroup', primaryjoin='TgPermission.permission_id==tg_group_permission.c.permission_id', secondary=tg_group_permission, secondaryjoin='tg_group_permission.c.group_id==TgGroup.group_id') 146 | 147 | 148 | class TgTown(DeclarativeBase): 149 | __tablename__ = 'tg_town' 150 | 151 | #column definitions 152 | town_id = Column(u'town_id', INTEGER(), primary_key=True, nullable=False) 153 | town_name = Column(u'town_name', VARCHAR(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 154 | 155 | #relation definitions 156 | 157 | 158 | class TgUser(DeclarativeBase): 159 | __tablename__ = 'tg_user' 160 | 161 | #column definitions 162 | created = Column(u'created', TIMESTAMP(timezone=False)) 163 | display_name = Column(u'display_name', VARCHAR(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 164 | email_address = Column(u'email_address', VARCHAR(length=255, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 165 | password = Column(u'password', VARCHAR(length=80, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 166 | town_id = Column(u'town_id', INTEGER(), ForeignKey('tg_town.town_id')) 167 | user_id = Column(u'user_id', INTEGER(), primary_key=True, nullable=False) 168 | user_name = Column(u'user_name', VARCHAR(length=16, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 169 | 170 | #relation definitions 171 | tg_town = relation('TgTown', primaryjoin='TgUser.town_id==TgTown.town_id') 172 | tg_groups = relation('TgGroup', primaryjoin='TgUser.user_id==tg_user_group.c.user_id', secondary=tg_user_group, secondaryjoin='tg_user_group.c.group_id==TgGroup.group_id') 173 | 174 | 175 | #example on how to query your Schema 176 | from sqlalchemy.orm import sessionmaker 177 | session = sessionmaker(bind=engine)() 178 | objs = session.query(TgGroup).all() 179 | print 'All TgGroup objects: %s'%objs""" 180 | assert expected in r, r 181 | 182 | 183 | class TestModelFactoryNew: 184 | 185 | def setup(self): 186 | self.metadata = make_test_db() 187 | engine = self.metadata.bind 188 | self.config = DummyConfig(engine) 189 | self.factory = ModelFactory(self.config) 190 | self.factory.models 191 | 192 | def test_tables(self): 193 | tables = sorted([t.name for t in self.factory.tables]) 194 | eq_(tables, [u'environment', u'no_pk', u'report', u'ui_report']) 195 | 196 | def test_setup_all_models(self): 197 | assert len(self.factory.models) == 3 198 | 199 | def test_repr_environ_model(self): 200 | print self.factory.models 201 | s = self.factory.models[0].__repr__() 202 | assert s == """class Environment(DeclarativeBase): 203 | __tablename__ = 'environment' 204 | 205 | __table_args__ = {} 206 | 207 | #column definitions 208 | database_host = Column(u'database_host', VARCHAR(length=100, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 209 | database_pass = Column(u'database_pass', VARCHAR(length=100, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 210 | database_port = Column(u'database_port', VARCHAR(length=5, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 211 | database_sid = Column(u'database_sid', VARCHAR(length=32, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 212 | database_user = Column(u'database_user', VARCHAR(length=100, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 213 | environment_id = Column(u'environment_id', NUMERIC(precision=10, scale=0, asdecimal=True), primary_key=True, nullable=False) 214 | environment_name = Column(u'environment_name', VARCHAR(length=100, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 215 | 216 | #relation definitions 217 | reports = relation('Report', primaryjoin='Environment.environment_id==UiReport.environment_id', secondary=ui_report, secondaryjoin='UiReport.report_id==Report.report_id') 218 | """, s 219 | 220 | def test_no_pk_table_in_output(self): 221 | s = self.factory.__repr__() 222 | assert """no_pk = Table(u'no_pk', metadata, 223 | Column(u'data', TEXT(length=None, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)), 224 | )""" in s, s 225 | 226 | 227 | def test_repr_report(self): 228 | s = self.factory.models[1].__repr__() 229 | assert s == """class Report(DeclarativeBase): 230 | __tablename__ = 'report' 231 | 232 | __table_args__ = {} 233 | 234 | #column definitions 235 | created_by = Column(u'created_by', NUMERIC(precision=10, scale=0, asdecimal=True), nullable=False) 236 | created_date = Column(u'created_date', DATETIME(timezone=False), nullable=False) 237 | deleted = Column(u'deleted', NUMERIC(precision=1, scale=0, asdecimal=True), nullable=False) 238 | deleted_by = Column(u'deleted_by', NUMERIC(precision=10, scale=0, asdecimal=True)) 239 | deleted_date = Column(u'deleted_date', DATETIME(timezone=False)) 240 | environment_id = Column(u'environment_id', NUMERIC(precision=10, scale=0, asdecimal=True), ForeignKey('environment.environment_id'), nullable=False) 241 | report_description = Column(u'report_description', VARCHAR(length=4000, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 242 | report_id = Column(u'report_id', NUMERIC(precision=10, scale=0, asdecimal=True), primary_key=True, nullable=False) 243 | report_name = Column(u'report_name', VARCHAR(length=50, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), nullable=False) 244 | updated_by = Column(u'updated_by', NUMERIC(precision=10, scale=0, asdecimal=True), nullable=False) 245 | updated_date = Column(u'updated_date', DATETIME(timezone=False), nullable=False) 246 | 247 | #relation definitions 248 | environment = relation('Environment', primaryjoin='Report.environment_id==Environment.environment_id') 249 | environments = relation('Environment', primaryjoin='Report.report_id==UiReport.report_id', secondary=ui_report, secondaryjoin='UiReport.environment_id==Environment.environment_id') 250 | """, s 251 | 252 | 253 | class TestModelFactoryMulti: 254 | 255 | def __init__(self, *args, **kw): 256 | 257 | self.metadata = make_test_db_multi() 258 | engine = self.metadata.bind 259 | self.config = DummyConfig(engine) 260 | self.factory = ModelFactory(self.config) 261 | # self.factory.models 262 | 263 | 264 | def test_get_foreign_keys(self): 265 | fks = [t.name for t in self.factory.get_foreign_keys(self.metadata.tables['song']).keys()] 266 | 267 | eq_(fks, ['album']) 268 | 269 | def test_get_composite_fks(self): 270 | fks = sorted([k.column.name for k in self.factory.get_composite_foreign_keys(self.metadata.tables['song'])[0]]) 271 | eq_(fks, [u'albumartist', u'albumname'] ) 272 | 273 | def test_render_song(self): 274 | self.factory.models 275 | song = self.factory.models[1] 276 | eq_(song.__repr__(), """class Song(DeclarativeBase): 277 | __tablename__ = 'song' 278 | 279 | __table_args__ = {} 280 | 281 | #column definitions 282 | songalbum = Column(u'songalbum', TEXT(length=None, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), ForeignKey('album.albumname')) 283 | songartist = Column(u'songartist', TEXT(length=None, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False), ForeignKey('album.albumartist')) 284 | songid = Column(u'songid', INTEGER(), primary_key=True) 285 | songname = Column(u'songname', TEXT(length=None, convert_unicode=False, assert_unicode=None, unicode_error=None, _warn_on_bytestring=False)) 286 | 287 | #relation definitions 288 | album = relation('Album', primaryjoin="and_(Song.songartist==Album.albumartist, Song.songalbum==Album.albumname)") 289 | """ 290 | ) 291 | 292 | -------------------------------------------------------------------------------- /sqlautocode/util.py: -------------------------------------------------------------------------------- 1 | import config 2 | import fnmatch 3 | import os 4 | import re 5 | import sys 6 | 7 | 8 | _defaultencoding = sys.getdefaultencoding() 9 | _python_identifier_re = re.compile(r'^[a-z_][a-z0-9_]*$', re.I) 10 | 11 | 12 | def emit(*lines): 13 | """Emit one or more output strings.""" 14 | 15 | for line in lines: 16 | if not line: 17 | config.out.write(os.linesep) 18 | else: 19 | if isinstance(line, unicode): 20 | encoding = 'utf-8' 21 | if getattr(config, 'options', None): 22 | encoding = config.options.encoding 23 | line = line.encode(encoding) 24 | config.out.write(line) 25 | if line[-1] != '\n': 26 | config.out.write(os.linesep) 27 | 28 | 29 | def is_python_identifier(string): 30 | """True if string is a valid Python identifier.""" 31 | 32 | # unicode-ok. 33 | return _python_identifier_re.match(string) 34 | 35 | 36 | def as_out_str(obj): 37 | """Like str(), but convert unicode to configured encoding.""" 38 | 39 | if isinstance(obj, unicode): 40 | encoding = 'utf-8' 41 | if getattr(config, 'options', None): 42 | encoding = config.options.encoding 43 | return obj.encode(encoding) 44 | elif not isinstance(obj, str): 45 | return str(obj) 46 | else: 47 | return obj 48 | 49 | 50 | def as_sys_str(obj, escape='backslashreplace'): 51 | """Like str(), but safely convert unicode to the system encoding.""" 52 | 53 | if isinstance(obj, unicode): 54 | return obj.encode(_defaultencoding, escape) 55 | elif not isinstance(obj, str): 56 | return str(obj) 57 | else: 58 | return obj 59 | 60 | 61 | def unique(iterable): 62 | seen = set() 63 | for item in iterable: 64 | if item not in seen: 65 | seen.add(item) 66 | yield item 67 | 68 | 69 | def glob_intersection(collection, subset): 70 | """Return elements of subset in collection, with glob support. 71 | 72 | collection 73 | A collection of strings, need not be a set. 74 | subset 75 | Any iterable of strings. 76 | 77 | Items in the subset may be plain strings, "quoted strings" or 78 | strings with*glob? characters. Quoted strings are not globbed. 79 | """ 80 | 81 | found, missing, unmatched = [], [], [] 82 | for identifier in unique(subset): 83 | if identifier[0] == '"': 84 | name = identifier[1:-1] 85 | if name in collection: 86 | found.append(name) 87 | else: 88 | missing.append(name) 89 | elif '*' not in identifier: 90 | if identifier in collection: 91 | found.append(identifier) 92 | else: 93 | missing.append(identifier) 94 | else: 95 | globbed = fnmatch.filter(collection, identifier) 96 | if globbed: 97 | found.extend(globbed) 98 | else: 99 | unmatched.append(identifier) 100 | 101 | # ordered sets sure would be nice. 102 | return list(unique(found)), missing, unmatched 103 | 104 | # lifted from http://www.daniweb.com/forums/thread70647.html 105 | # (pattern, search, replace) regex english plural rules tuple 106 | plural_rule_tuple = ( 107 | ('[ml]ouse$', '([ml])ouse$', '\\1ice'), 108 | ('child$', 'child$', 'children'), 109 | ('booth$', 'booth$', 'booths'), 110 | ('foot$', 'foot$', 'feet'), 111 | ('ooth$', 'ooth$', 'eeth'), 112 | ('l[eo]af$', 'l([eo])af$', 'l\\1aves'), 113 | ('sis$', 'sis$', 'ses'), 114 | ('man$', 'man$', 'men'), 115 | ('ife$', 'ife$', 'ives'), 116 | ('eau$', 'eau$', 'eaux'), 117 | ('lf$', 'lf$', 'lves'), 118 | ('[xz]$', '$', 'es'), 119 | ('[s]$', '$', ''), 120 | ('[^aeioudgkprt]h$', '$', 'es'), 121 | ('(qu|[^aeiou])y$', 'y$', 'ies'), 122 | ('$', '$', 's') 123 | ) 124 | 125 | singular_rule_tuple = ( 126 | ('[ml]ouse$', '([ml])ouse$', '\\1ice'), 127 | ('children$', 'children$', 'child'), 128 | ('feet$', 'fee$', 'foot'), 129 | ('eeth$', 'eeth$', 'ooth'), 130 | ('l[eo]aves', 'l([eo])af$', 'l\\1af$'), 131 | ('ses$', 'ses$', 's'), 132 | ('men$', 'men$', 'man'), 133 | ('ives$', 'ives$', 'ife'), 134 | ('eaux$', 'eaux$', 'eau'), 135 | ('lves$', 'lves$', 'lf'), 136 | # ('[xz]$', '$', 'es'), not sure how to unplural this one 137 | # ('[s]$', '$', ''), 138 | ('pies$', 'pies$', 'pie'), 139 | ('ovies$', 'ovies$', 'ovie'), 140 | ('ies$', 'ies$', 'y'), 141 | ('xes$', 'xes$', 'x'), 142 | # ('(qu|[^aeiou])y$', 'y$', 'ies'), 143 | ('s$', 's$', '') 144 | ) 145 | 146 | 147 | def regex_rules(rules): 148 | for line in rules: 149 | pattern, search, replace = line 150 | yield lambda word: re.search(pattern, word) and re.sub(search, replace, word) 151 | 152 | plural_rules = regex_rules(plural_rule_tuple) 153 | 154 | 155 | def plural(noun): 156 | for rule in regex_rules(plural_rule_tuple): 157 | result = rule(noun) 158 | if result: 159 | return result 160 | return noun 161 | 162 | 163 | def singular(noun): 164 | for rule in regex_rules(singular_rule_tuple): 165 | result = rule(noun) 166 | if result: 167 | return result 168 | return noun 169 | 170 | 171 | def name2label(name, schema=None): 172 | """ 173 | Convert a column name to a Human Readable name. 174 | borrowed from old TG fastdata code 175 | """ 176 | # Create label from the name: 177 | # 1) Convert _ to Nothing 178 | # 2) Convert CamelCase to Camel Case 179 | # 3) Upcase first character of Each Word 180 | # Note: I *think* it would be thread-safe to 181 | # memoize this thing. 182 | if schema: 183 | if name.startswith(schema + '.'): 184 | name = '.'.join(name.split('.')[1:]) 185 | label = str(''.join([s.capitalize() for s in 186 | re.findall(r'([A-Z][a-z0-9]+|[a-z0-9]+|[A-Z0-9]+)', name)])) 187 | return label 188 | --------------------------------------------------------------------------------