├── LICENSE ├── README.rst ├── publication.py └── pyproject.toml /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2018 Glyph 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | What is this? 2 | ============= 3 | 4 | Setting expectations around what APIs you can rely on in a Python 5 | library is very difficult. 6 | 7 | Publication makes it easy. 8 | 9 | The Problem 10 | ----------- 11 | 12 | As `Hyrum's Law `_ somewhat grimly states, 13 | 14 | | With a sufficient number of users of an API, 15 | | it does not matter what you promise in the contract: 16 | | all observable behaviors of your system 17 | | will be depended on by somebody. 18 | 19 | In general, Python famously has a somewhat different philosophical view of this 20 | reality. We assume each other to be `responsible users 21 | `_ of the libraries 22 | we consume. Mucking with implementation details might break every time you 23 | upgrade, but it's sufficiently useful for testing, debugging, and 24 | experimentation that retaining that ability is worth paying the cost. 25 | 26 | But, critical to this assumption is that everybody *knows* when they're 27 | breaking into the "private" area of the library's interface. Here, there's a 28 | mismatch of expectations: 29 | 30 | - *library authors* write documentation, and then think that users sit down and 31 | read the documentation, front to back, and learn about what the "public" 32 | interface is by doing so. they then assume that users will know that they've 33 | used private implementation details if they ever deviate from these 34 | documented features. 35 | - *library users* ``pip install`` a thing, open up a REPL, import the module, 36 | and discover the library by doing ``dir()`` on the module and its contents, 37 | assuming that their program is not using any private implementation details 38 | as long as they never had to type ``library._something_private()`` while 39 | doing so. If they ever encounter a traceback they may consult the 40 | documentation, briefly, until it is resolved. 41 | 42 | Publication makes it possible to align the wildly divergent expectations of 43 | these groups, so that users can still get the benefits of being able to use 44 | internal details if they want, but they'll know that they're doing so. It 45 | makes the runtime namespace of your module look like the public documentation 46 | of your library. 47 | 48 | How does this look in practice? 49 | ------------------------------- 50 | 51 | You, a prospective library author, want to write a library that makes it easy to zorf a sprocket. 52 | Great! You do, and it looks like this: 53 | 54 | .. code:: python 55 | 56 | # sprocket_zorfer.py 57 | from sprocket import sprocket_with_name 58 | from zorf import zorfable_thing 59 | 60 | def zorf_sprocket_internal(sprocket, zorfulations): 61 | ... 62 | def compute_zorfulations(): 63 | ... 64 | 65 | def zorf_sprocket_named(sprocket_name, how_much): 66 | sprocket = sprocket_with_name(sprocket_name) 67 | zorfulations = compute_zorfulations(how_much) 68 | return zorf_sprocket_internal(sprocket, zorfulations) 69 | 70 | 71 | __all__ = [ 72 | 'zorf_sprocket_named' 73 | ] 74 | 75 | Your intent here, of course, is that you have exposed a module with a 76 | single function: ``zorf_sprocket_named``, and everything else is an 77 | implementation detail. You even said so, explicitly, with ``__all__``. 78 | Your API documentation says the same. 79 | 80 | However, reading reference documentation and cleanly respecting 81 | conventions is not how working programmers really figure out how to use 82 | stuff. Your users all do stuff like: 83 | 84 | - Load up an interactive ``python`` interpreter and call ``dir()`` on 85 | your module 86 | - Install Jupyter and tab-complete their way around your module to find 87 | what they want 88 | - use the auto-import function in PyCharm to grab some private 89 | implementation detail 90 | 91 | and, before you know it, you have thousands of users of your library 92 | with code like 93 | 94 | .. code:: python 95 | 96 | from sprocket_zorfer import compute_zorfulations, zorf_sprocket_internal, sprocket_with_name 97 | 98 | sprocket = sprocket_with_name(name) 99 | zorf_sprocket_internal(sprocket, compute_zorfulations(7) * 2) 100 | 101 | Now you can never change *any* of your implementation details! Worse 102 | yet, ``sprocket_with_name`` isn’t even your own code; that’s something 103 | you got from a library! But when someone does 104 | ``import sprocket_zorfer; sprocket_zorfer.`` in an interactive 105 | shell, none of that information comes through. 106 | 107 | Underscore Paranoia 108 | ------------------- 109 | 110 | The convention in Python is that we use ``_`` to indicate private names. 111 | So when we library authors notice this problem starting to happen, a 112 | common reaction is to start putting ``_`` in front of *everything* – 113 | class names, function names, module names – and only explicitly export 114 | those things that should be public by “moving” them via an import and an 115 | entry in a public module’s ``__all__``. 116 | 117 | However, this has a bunch of disadvantages: 118 | 119 | - Most code inspection tooling and IDEs won’t see that the public name 120 | is “moved”, so code exploration just makes it seem like *everything* 121 | is an implementation detail now, rather than making it seem like 122 | nothing is. 123 | 124 | - All your ``__repr__``\ s now have ugly and inaccurate function and 125 | class names in them, at least from the perspective of your users; how 126 | are they supposed to know ``zorf_sprocket_named`` is actually defined 127 | in ``zorf_sprocket._impl_details.funcs._zorf_sprocket_public`` now? 128 | How are they supposed to find the good, public name once they’re 129 | looking at the goofy internal one? 130 | 131 | - You constantly need to remember to put *all* of your code in these 132 | ugly ``_``-prefixed modules, and educate new contributors as to the 133 | risks of creating new modules in your package that are not carefully 134 | hidden away from public users. 135 | 136 | A Better World 137 | -------------- 138 | 139 | What if you could write all your code *as if* it were just regular 140 | public code, and have all your implementation details and imports 141 | automatically squirreled away in an underscore namespace so that curious 142 | coders won’t accidentally find every module you ever imported and every 143 | temporary helper function you ever defined and think they’re part of the 144 | permanent public face of your library? 145 | 146 | Enter ``publication``. 147 | 148 | ``publication`` uses the existing convention of ``__all__`` and a little 149 | runtime hackery to hide everything that you have not marked as 150 | explicitly public, like so: 151 | 152 | .. code:: python 153 | 154 | # sprocket_zorfer.py 155 | 156 | from publication import publish 157 | 158 | from sprocket import sprocket_with_name 159 | from zorf import zorfable_thing 160 | 161 | def zorf_sprocket_internal(sprocket, zorfulations): 162 | ... 163 | def compute_zorfulations(): 164 | ... 165 | 166 | def zorf_sprocket_named(sprocket_name, how_much): 167 | sprocket = sprocket_with_name(sprocket_name) 168 | zorfulations = compute_zorfulations(how_much) 169 | return zorf_sprocket_internal(sprocket, zorfulations) 170 | 171 | 172 | __all__ = [ 173 | 'zorf_sprocket_named' 174 | ] 175 | 176 | publish() 177 | 178 | That’s it! Now, ``from sprocket_zorfer import zorf_sprocket_named`` 179 | works as intended, but 180 | ``from sprocket_zorfer import compute_zorfulations`` is an 181 | ``ImportError``. 182 | 183 | But what about… 184 | --------------- 185 | 186 | Other modules in my package, like tests, that need to peek at implementation details? 187 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 188 | 189 | Don’t worry, your code didn’t go anywhere. The original module is still 190 | available as a special pseudo-module called ``._private``. 191 | In the example above, ``sprocket_zorfer.py``\ ’s tests can still do: 192 | 193 | .. code:: python 194 | 195 | from sprocket_zorfer._private import compute_zorfulations 196 | 197 | def test_compute_zorfulations(): 198 | assert compute_zorfulations(0) > 7 199 | 200 | Mypy? 201 | ~~~~~ 202 | 203 | Your types should *probably* just be part of your published API, if 204 | you’re expecting that users will need to know about them. But, if there 205 | are cases which need to be type-checked internally in your library, as 206 | far as Mypy is concerned, all your private classes are still there. So, 207 | in the simple case you can just do this: 208 | 209 | .. code:: python 210 | 211 | from typing import TYPE_CHECKING 212 | if TYPE_CHECKING: 213 | from something import T 214 | 215 | def returns_a() -> "T": 216 | ... 217 | 218 | and in the hopefully very unusual case you need to mix runtime and 219 | type-checking access to a different module’s private details, 220 | 221 | .. code:: python 222 | 223 | from typing import TYPE_CHECKING 224 | if TYPE_CHECKING: 225 | from something import T 226 | else: 227 | from something._private import T 228 | 229 | def returns_a() -> A: 230 | ... 231 | -------------------------------------------------------------------------------- /publication.py: -------------------------------------------------------------------------------- 1 | 2 | """ 3 | Publication helps you maintain public-api-friendly modules by preventing \ 4 | unintentional access to private implementation details via introspection. 5 | 6 | It's easy to use:: 7 | 8 | # yourmodule.py 9 | import dependency1 10 | import dependency2 11 | 12 | from publication import publish 13 | 14 | def implementation_detail(): 15 | ... 16 | 17 | def stuff(): 18 | ... 19 | implementation_detail() 20 | ... 21 | 22 | __all__ = [ 23 | 'stuff' 24 | ] 25 | 26 | publish() 27 | 28 | Now, C{from yourmodule import dependency1} just raises an C{ImportError} - as 29 | you would want; C{dependency1} isn't part of yourmodule! So does C{from 30 | yourmodule import dependency1} Only C{stuff} is I{supposed} to be in the public 31 | interface you're trying to support, so only it can be imported. 32 | 33 | All your implementation details are still accessible in a namespace called 34 | C{_private}, which you can still use via C{from yourmodule._private import 35 | dependency1}, for white-box testing and similar use-cases. 36 | """ 37 | 38 | from types import ModuleType 39 | import sys 40 | 41 | PRIVATE_NAME = "_private" 42 | 43 | _nothing = object() 44 | 45 | 46 | def publish(): 47 | # type: () -> None 48 | """ 49 | Publish the interface of the calling module as defined in C{__all__}; 50 | relegate the rest of it to a C{_private} API module. 51 | 52 | Call it at the top level of your module after C{__all__} and all the names 53 | described in it are defined; usually the best place to do this is as the 54 | module's last line. 55 | """ 56 | localvars = sys._getframe(1).f_locals 57 | name = localvars["__name__"] 58 | all = localvars["__all__"] 59 | public = ModuleType(name) 60 | private = sys.modules[name] 61 | sys.modules[name] = public 62 | names = all + [ 63 | "__all__", 64 | "__cached__", 65 | "__doc__", 66 | "__file__", 67 | "__loader__", 68 | "__name__", 69 | "__package__", 70 | "__path__", 71 | "__spec__", 72 | ] 73 | for published in names: 74 | value = getattr(private, published, _nothing) 75 | if value is not _nothing: 76 | setattr(public, published, value) 77 | setattr(public, PRIVATE_NAME, private) 78 | sys.modules[".".join([name, PRIVATE_NAME])] = private 79 | 80 | 81 | __version__ = "0.0.3" 82 | 83 | __all__ = ["publish", "__version__"] 84 | publish() 85 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["flit"] 3 | build-backend = "flit.buildapi" 4 | 5 | [tool.flit.metadata] 6 | module = "publication" 7 | author = "Glyph" 8 | author-email = "glyph@twistedmatrix.com" 9 | home-page = "https://github.com/glyph/publication" 10 | description-file = "README.rst" 11 | classifiers = ["License :: OSI Approved :: MIT License", 12 | "Programming Language :: Python :: 3", 13 | "Programming Language :: Python :: 2"] 14 | --------------------------------------------------------------------------------