STYLE.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Elements of Python Style\n",
    "\n",
    "This document goes beyond PEP8 to cover the core of what I think of as great Python style. It is opinionated, but not too opinionated. It goes beyond mere issues of syntax and module layout, and into areas of paradigm, organization, and architecture. I hope it can be a kind of condensed [\"Strunk & White\"][strunk-white] for Python code.\n",
    "\n",
    "With credit and respect to [the original author](https://github.com/amontalenti/elements-of-python-style), I've included this here for my own reference. I may also expand upon it.\n",
    "\n",
    "[strunk-white]: https://en.wikipedia.org/wiki/The_Elements_of_Style\n",
    "\n",
    "# Table of Contents\n",
    "\n",
    "  * [The Elements of Python Style](#the-elements-of-python-style)\n",
    "    * [Follow Most PEP8 Guidelines](#follow-most-pep8-guidelines)\n",
    "    * [Flexibility on Line Length](#flexibility-on-line-length)\n",
    "    * [Consistent Naming](#consistent-naming)\n",
    "    * [Nitpicks That Aren't Worth It](#nitpicks-that-arent-worth-it)\n",
    "    * [Writing Good Docstrings](#writing-good-docstrings)\n",
    "    * [Paradigms and Patterns](#paradigms-and-patterns)\n",
    "    * [A Little Zen for Your Code Style](#a-little-zen-for-your-code-style)\n",
    "    * [Six of One, Half a Dozen of the Other](#six-of-one-half-a-dozen-of-the-other)\n",
    "    * [Standard Tools and Project Structure](#standard-tools-and-project-structure)\n",
    "    * [Some Inspiration](#some-inspiration)\n",
    "    * [Contributors](#contributors)\n",
    "\n",
    "## Follow Most [PEP8 Guidelines][pep8]\n",
    "\n",
    "... but, be flexible on naming and line length.\n",
    "\n",
    "PEP8 covers lots of mundane stuff like whitespace, line breaks between functions/classes/methods, imports, and warning against use of deprecated functionality. Pretty much everything in there is good.\n",
    "\n",
    "The best tool to enforce these rules, while also helping you catch silly Python syntax errors, is [flake8][flake8].\n",
    "\n",
    "PEP8 is meant as a set of guidelines, not rules to be strictly, or religiously, followed. Make sure to read the section of PEP8 that is titled: \"A Foolish Consistency is the Hobgoblin of Little Minds.\" Also see Raymond Hettinger's excellent talk, [\"Beyond PEP8\"](https://www.youtube.com/watch?v=wf-BqAjZb8M) for more on this.\n",
    "\n",
    "The only set of rules that seem to cause a disproportionate amount of controversy are around the line length and naming. These can be easily tweaked.\n",
    "\n",
    "## Flexibility on Line Length\n",
    "\n",
    "If the strict 79-character line length rule in `flake8` bothers you, feel free to ignore or adjust that rule. It's probably still a good rule-of-thumb -- like a \"rule\" that says English sentences should have 50 or fewer words, or that paragraphs should have fewer than 10 sentences. Here's the link to [flake8 config][f8config], see the `max-line-length` config option. Note also that often a `# noqa` comment can be added to a line to have a `flake8` check ignored, but please use these sparingly.\n",
    "\n",
    "90%+ of your lines should be 79 characters or fewer, though, for the simple reason that \"Flat is better than nested\". If you find a function where all the lines are longer than this, something else is wrong, and you should look at your code rather than at your flake8 settings.\n",
    "\n",
    "[pep8]: https://www.python.org/dev/peps/pep-0008/\n",
    "[flake8]: https://flake8.readthedocs.org\n",
    "[f8config]: https://flake8.readthedocs.org/en/latest/config.html\n",
    "\n",
    "## Consistent Naming\n",
    "\n",
    "On naming, following some simple rules can prevent a whole lot of team-wide grief.\n",
    "\n",
    "### Preferred Naming Rules\n",
    "\n",
    "Many of these were adapted from [the Pocoo team][pocoo].\n",
    "\n",
    "- Class names: `CamelCase`, and capitalize acronyms: `HTTPWriter`, not `HttpWriter`.\n",
    "- Variable names: `lower_with_underscores`.\n",
    "- Method and function names: `lower_with_underscores`.\n",
    "- Modules: `lower_with_underscores.py`. (But, prefer names that don't need underscores!)\n",
    "- Constants: `UPPER_WITH_UNDERSCORES`.\n",
    "- Precompiled regular expressions: `name_re`.\n",
    "\n",
    "[pocoo]: http://www.pocoo.org/internal/styleguide/\n",
    "\n",
    "You should generally follow these rules, unless you are mirroring some other tool's naming convention, like a database schema or message format.\n",
    "\n",
    "You can also choose to use `CamelCase` for things that are class-like but not quite classes -- the main benefit of `CamelCase` is calling attention to something as a \"global noun\", rather than a local label or a verb. Notice that Python names `True`, `False`, and `None` use `CamelCase` even though they are not classes.\n",
    "\n",
    "### Avoid Name Adornments\n",
    "\n",
    "... like `_prefix` or `suffix_`. Functions and methods can have a `_prefix` notation to indicate \"private\", but this should be used sparingly and only for APIs that are expected to be widely used, and where the `_private` indicator assists with [information hiding][infohiding].\n",
    "\n",
    "[infohiding]: http://c2.com/cgi/wiki?InformationHiding\n",
    "\n",
    "PEP8 suggests using a trailing underscore to avoid aliasing a built-in, e.g.\n",
    "\n",
    "```python\n",
    "sum_ = sum(some_long_list)\n",
    "print(sum_)\n",
    "```\n",
    "\n",
    "This is OK in a pinch, but it might be better to just choose a different name.\n",
    "\n",
    "You should rarely use `__mangled` double-underscore prefixes for class/instance/method labels, which have special [name mangling behavior][mangling] -- it's rarely necessary. Never create your own names using `__dunder__` adornments unless you are implementing a Python standard protocol, like `__len__`; this is a namespace specifically reserved for Python's internal protocols and shouldn't be co-opted for your own stuff.\n",
    "\n",
    "[mangling]: https://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references\n",
    "\n",
    "### Avoid One-Character Names\n",
    "\n",
    "There are some one-character label names that are common and acceptable.\n",
    "\n",
    "With `lambda`, using `x` for single-argument functions is OK. For example:\n",
    "\n",
    "```python\n",
    "encode = lambda x: x.encode(\"utf-8\", \"ignore\")\n",
    "```\n",
    "\n",
    "With tuple unpacking, using `_` as a throwaway label is also OK. For example:\n",
    "\n",
    "```python\n",
    "_, url, urlref = data\n",
    "```\n",
    "\n",
    "This basically means, \"ignore the first element.\"\n",
    "\n",
    "Similar to `lambda`, inside list/dict/set comprehensions, generator expressions, or very short (1-2 line) for loops, a single-char iteration label can be used. This is also typically `x`, e.g.\n",
    "\n",
    "```python\n",
    "sum(x for x in items if x > 0)\n",
    "```\n",
    "\n",
    "to sum all positive integers in the sequence `items`.\n",
    "\n",
    "It is also very common to use `i` as shorthand for \"index\", and commonly with the `enumerate` built-in. For example:\n",
    "\n",
    "```python\n",
    "for i, item in enumerate(items):\n",
    "    print(\"%4s: %s\" % (i, item))\n",
    "```\n",
    "\n",
    "Outside of these cases, you should rarely, perhaps **never**, use single-character label/argument/method names. This is because it just makes it impossible to `grep` for stuff.\n",
    "\n",
    "### Use `self` and similar conventions\n",
    "\n",
    "You should:\n",
    "\n",
    "- always name a method's first argument `self`\n",
    "- always name `@classmethod`'s first argument `cls`\n",
    "- always use `*args` and `**kwargs` for variable argument lists\n",
    "\n",
    "## Nitpicks That Aren't Worth It\n",
    "\n",
    "There's nothing to gain from not following these rules, so you should just follow them.\n",
    "\n",
    "### Always [inherit from `object`][newstyle] and use new-style classes\n",
    "\n",
    "```python\n",
    "# bad\n",
    "class JSONWriter:\n",
    "    pass\n",
    "\n",
    "# good\n",
    "class JSONWriter(object):\n",
    "    pass\n",
    "```\n",
    "\n",
    "In Python 2, it's important to follow this rule. In Python 3, all classes implicitly inherit from `object` and this rule isn't necessary any longer.\n",
    "\n",
    "### Don't repeat instance labels in the class\n",
    "\n",
    "```python\n",
    "# bad\n",
    "class JSONWriter(object):\n",
    "    handler = None\n",
    "    def __init__(self, handler):\n",
    "        self.handler = handler\n",
    "\n",
    "# good\n",
    "class JSONWriter(object):\n",
    "    def __init__(self, handler):\n",
    "        self.handler = handler\n",
    "```\n",
    "\n",
    "### Prefer [list/dict/set comprehensions][mapfilter] over map/filter.\n",
    "\n",
    "```python\n",
    " # bad\n",
    "map(truncate, filter(lambda x: len(x) > 30, items))\n",
    "\n",
    " # good\n",
    "[truncate(x) for x in items if len(x) > 30]\n",
    "```\n",
    "\n",
    "Though you should prefer comprehensions for most of the simple cases, there are occasions where `map()` or `filter()` will be more readable, so use your judgment.\n",
    "\n",
    "### Use parens `(...)` for continuations\n",
    "\n",
    "```python\n",
    "# bad\n",
    "from itertools import groupby, chain, \\\n",
    "                      izip, islice\n",
    "\n",
    "# good\n",
    "from itertools import (groupby, chain,\n",
    "                       izip, islice)\n",
    "```\n",
    "\n",
    "### Use parens `(...)` for fluent APIs\n",
    "\n",
    "```python\n",
    "# bad\n",
    "response = Search(using=client) \\\n",
    "           .filter(\"term\", cat=\"search\") \\\n",
    "           .query(\"match\", title=\"python\")\n",
    "\n",
    "# good\n",
    "response = (Search(using=client)\n",
    "            .filter(\"term\", cat=\"search\")\n",
    "            .query(\"match\", title=\"python\"))\n",
    "```\n",
    "\n",
    "### Use implicit continuations in function calls\n",
    "\n",
    "```python\n",
    "# bad -- simply unnecessary backslash\n",
    "return set((key.lower(), val.lower()) \\\n",
    "           for key, val in mapping.iteritems())\n",
    "\n",
    "# good\n",
    "return set((key.lower(), val.lower())\n",
    "           for key, val in mapping.iteritems())\n",
    "```\n",
    "\n",
    "### Use `isinstance(obj, cls)`, not `type(obj) == cls`\n",
    "\n",
    "This is because `isinstance` covers way more cases, including sub-classes and ABC's. Also, rarely use `isinstance` at all, since you should usually be doing duck typing, instead!\n",
    "\n",
    "### Use `with` for files and locks\n",
    "\n",
    "The `with` statement subtly handles file closing and lock releasing even in the case of exceptions being raised. So:\n",
    "\n",
    "```python\n",
    "# bad\n",
    "somefile = open(\"somefile.txt\", \"w\")\n",
    "somefile.write(\"sometext\")\n",
    "return\n",
    "\n",
    "# good\n",
    "with open(\"somefile.txt\", \"w\") as somefile:\n",
    "    somefile.write(\"sometext\")\n",
    "return\n",
    "```\n",
    "\n",
    "### Use `is` when comparing to `None`\n",
    "\n",
    "The `None` value is a singleton but when you're checking for `None`, you rarely want to actually call `__eq__` on the LHS argument. So:\n",
    "\n",
    "```python\n",
    "# bad\n",
    "if item == None:\n",
    "    continue\n",
    "\n",
    "# good\n",
    "if item is None:\n",
    "   continue\n",
    "```\n",
    "\n",
    "Not only is the good form faster, it's also more correct. It's no more concise to use `==`, so just remember this rule!\n",
    "\n",
    "### Avoid `sys.path` hacks\n",
    "\n",
    "It can be tempting to do `sys.path.insert(0, \"../\")` and similar to control Python's import approach, but you should avoid these like the plague.\n",
    "\n",
    "Python has a somewhat-complex, but very comprehensible, approach to module path resolution. You can adjust how Python loads modules via `PYTHONPATH` or via tricks like `setup.py develop`. You can also run Python using `-m` to good effect, e.g. `python -m mypkg.mymodule` rather than `python mypkg/mymodule.py`. You should not rely upon the current working directory that you run python out of for your code to work properly. David Beazley saves the day once more with his PDF slides which are worth a skim, [\"Modules and Packages: Live and Let Die!\"][modules]\n",
    "\n",
    "[modules]: http://www.dabeaz.com/modulepackage/ModulePackage.pdf\n",
    "\n",
    "### Rarely create [your own exception types][exceptiontypes]\n",
    "\n",
    "... and when you must, don't make too many.\n",
    "\n",
    "```python\n",
    " # bad\n",
    " class ArgumentError(Exception):\n",
    "     pass\n",
    " ...\n",
    " raise ArgumentError(url)\n",
    "\n",
    " # good\n",
    " raise ValueError(\"bad value for url: %s\" % url)\n",
    "```\n",
    "\n",
    "Note that Python includes [a rich set of built-in exception classes][ex-tree]. Leverage these appropriately, and you should \"customize\" them simply by instantiating them with string messages that describe the specific error condition you hit. It is most common to raise `ValueError` (bad argument), `LookupError` (bad key), or `AssertionError` (via the `assert` statement) in user code.\n",
    "\n",
    "A good rule of thumb for whether you should create your own exception type is to figure out whether a caller should catch it **every time** they call your function. If so, you probably **should** make your own type. But this is relatively rare. A good example of an exception type that clearly had to exist is [tornado.web.HTTPError][http-error]. But notice how Tornado did not go overboard: there is one exception class for **all** HTTP errors raised by the framework or user code.\n",
    "\n",
    "[ex-tree]: https://docs.python.org/2/library/exceptions.html#exception-hierarchy\n",
    "[http-error]: http://www.tornadoweb.org/en/stable/web.html#tornado.web.HTTPError\n",
    "\n",
    "### Short docstrings are proper one-line sentences\n",
    "\n",
    "```python\n",
    "# bad\n",
    "def reverse_sort(items):\n",
    "    \"\"\"\n",
    "    sort items in reverse order\n",
    "    \"\"\"\n",
    "\n",
    "# good\n",
    "def reverse_sort(items):\n",
    "    \"\"\"Sort items in reverse order.\"\"\"\n",
    "```\n",
    "\n",
    "Keep the triple-quote's on the same line `\"\"\"`, capitalize the first letter, and include a period. Four lines become two, the `__doc__` attribute doesn't have crufty newlines, and the pedants are pleased!\n",
    "\n",
    "### Use [reST for docstrings][docstrings]\n",
    "\n",
    "It's done by the stdlib and most open source projects. It's supported out-of-the-box by Sphinx. Just do it! The Python `requests` module uses these to extremely good effect. See the [`requests.api`][requests-api] module, for example.\n",
    "\n",
    "### Strip trailing whitespace\n",
    "\n",
    "This is perhaps the ultimate nitpick, but if you don't do it, it will drive people crazy. There are no shortage of tools that will do this for you in your text editor automatically; here's [a link to the one I use for vim][whitespace].\n",
    "\n",
    "[whitespace]: https://github.com/amontalenti/home/blob/master/.vim/bundle/whitespace/plugin/whitespace.vim\n",
    "\n",
    "## Writing Good Docstrings\n",
    "\n",
    "Here's a quick reference to using Sphinx-style reST in your function docstrings:\n",
    "\n",
    "```python\n",
    "def get(url, qsargs=None, timeout=5.0):\n",
    "    \"\"\"Send an HTTP GET request.\n",
    "\n",
    "    :param url: URL for the new request.\n",
    "    :type url: str\n",
    "    :param qsargs: Converted to query string arguments.\n",
    "    :type qsargs: dict\n",
    "    :param timeout: In seconds.\n",
    "    :rtype: mymodule.Response\n",
    "\"\"\"\n",
    "return request('get', url, qsargs=qsargs, timeout=timeout)\n",
    "```\n",
    "\n",
    "Don't document for the sake of documenting. The way to think about this is:\n",
    "\n",
    "```python\n",
    "good_names + explicit_defaults > verbose_docs + type_specs\n",
    "```\n",
    "\n",
    "That is, in the example above, there is no need to say `timeout` is a `float`, because the default value is `5.0`, which is clearly a `float`. It is useful to indicate in the documentation that the semantic meaning is \"seconds\", thus `5.0` means 5 seconds. Meanwhile, the caller has no clue what `qsargs` should be, so we give a hint with the `type` annotation, and the caller also has no clue what to expect back from the function, so an `rtype` annotation is appropriate.\n",
    "\n",
    "One last point. Guido once said that his key insight for Python is that, \"code is read much more often than it is written.\" Well, a corollary of this is that **some documentation helps, but too much documentation hurts**.\n",
    "\n",
    "You should basically only document functions you expect to be widely re-used. If you document every function in an internal module, you'll just end up with a less maintainable module, since the documentation needs to be refactored when the code is refactored. Don't \"cargo cult\" your docstrings and definitely don't auto-generate them with tooling!\n",
    "\n",
    "[newstyle]: https://docs.python.org/2/reference/datamodel.html#new-style-and-classic-classes\n",
    "[mapfilter]: http://www.artima.com/weblogs/viewpost.jsp?thread=98196\n",
    "[exceptiontypes]: https://twitter.com/amontalenti/status/665338326396727297\n",
    "[docstrings]: https://www.python.org/dev/peps/pep-0287/\n",
    "[requests-api]: https://github.com/kennethreitz/requests/blob/master/requests/api.py\n",
    "\n",
    "## Paradigms and Patterns\n",
    "\n",
    "### Functions vs classes\n",
    "\n",
    "You should usually prefer functions to classes. Functions and modules are the basic units of code re-use in Python, and they are the most flexible form. Classes are an \"upgrade path\" for certain Python facilities, such as implementing containers, proxies, descriptors, type systems, and more. But usually, functions are a better option.\n",
    "\n",
    "Some might like the code organization benefits of grouping related functions together into classes. But this is a mistake. You should group related functions together into **modules**.\n",
    "\n",
    "Though sometimes classes can act as a helpful \"mini namespace\" (e.g. with `@staticmethod`), more often a group of methods should be contributing to the internal operation of an object, rather than merely being a behavior grouping.\n",
    "\n",
    "It's always better to have a `lib.time` module for time-related functions than to have a `TimeHelper` class with a bunch of methods you are forced to subclass in order to use! Classes proliferate other classes, which proliferates complexity and decreases readability.\n",
    "\n",
    "### Generators and iterators\n",
    "\n",
    "Generators and iterators are Python's most powerful features -- you should master the iterator protocol, the `yield` keyword, and generator expressions.\n",
    "\n",
    "Not only are generators important for any function that needs to be called over a large stream of data, but they also have the effect of simplifying code by making it easy for you to write your own iterators. Refactoring code to generators often simplifies it while making it work in more scenarios.\n",
    "\n",
    "Luciano Ramalho, author of \"Fluent Python\", has a 30-minute presentation, [\"Iterators & Generators: the Python Way\"](https://www.youtube.com/watch?v=z4P6hSa6K9g), which gives an excellent, fast-paced overview. David Beazley, author of \"Python Essential Reference\" and \"Python Cookbook\", has a mind-bending three-hour video tutorial entitled [\"Generators: The Final Frontier\"](https://www.youtube.com/watch?v=5-qadlG7tWo) that is a satisfying exposition of generator use cases. Mastering this topic is worth it because it applies everywhere.\n",
    "\n",
    "### Declarative vs imperative\n",
    "\n",
    "You should prefer declarative to imperative programming. This is code that says  **what** you want to do, rather than code that describes **how** to do it. Python's [functional programming guide][func] includes some good details and examples of how to use this style effectively.\n",
    "\n",
    "[func]: https://docs.python.org/3/howto/functional.html\n",
    "\n",
    "You should use lightweight data structures like `list`, `dict`, `tuple`, and `set` to your advantage. It's always better to lay out your data, and then write some code to transform it, than to build up data by repeatedly calling mutating functions/methods.\n",
    "\n",
    "An example of this is the common list comprehension refactoring:\n",
    "\n",
    "```python\n",
    "# bad\n",
    "filtered = []\n",
    "for x in items:\n",
    "    if x.endswith(\".py\"):\n",
    "        filtered.append(x)\n",
    "return filtered\n",
    "```\n",
    "\n",
    "This should be rewritten as:\n",
    "\n",
    "```python\n",
    "# good\n",
    "return [x\n",
    "        for x in items\n",
    "        if x.endswith(\".py\")]\n",
    "```\n",
    "\n",
    "But another good example is rewriting an `if`/`elif`/`else` chain as a `dict` lookup.\n",
    "\n",
    "### Prefer \"pure\" functions and generators\n",
    "\n",
    "This is a concept that we can borrow from the functional programming community.  These kinds of functions and generators are alternatively described as \"side-effect free\", \"referentially transparent\", or as having \"immutable inputs/outputs\".\n",
    "\n",
    "As a simple example, you should avoid code like this:\n",
    "\n",
    "```python\n",
    "# bad\n",
    "def dedupe(items):\n",
    "    \"\"\"Remove dupes in-place, return items and # of dupes.\"\"\"\n",
    "    seen = set()\n",
    "    dupe_positions = []\n",
    "    for i, item in enumerate(items):\n",
    "        if item in seen:\n",
    "            dupe_positions.append(i)\n",
    "        else:\n",
    "            seen.add(item)\n",
    "    num_dupes = len(dupe_positions)\n",
    "    for idx in reversed(dupe_positions):\n",
    "        items.pop(idx)\n",
    "    return items, num_dupes\n",
    "```\n",
    "\n",
    "This same function can be written as follows:\n",
    "\n",
    "```python\n",
    "# good\n",
    "def dedupe(items):\n",
    "    \"\"\"Return deduped items and # of dupes.\"\"\"\n",
    "    deduped = set(items)\n",
    "    num_dupes = len(items) - len(deduped)\n",
    "    return deduped, num_dupes\n",
    "```\n",
    "\n",
    "This is a somewhat shocking example. In addition to making this function pure, we also made it much, much shorter. It's not only shorter: it's better. Its purity means `assert dedupe(items) == dedupe(items)` always holds true for the \"good\" version. In the \"bad\" version, `num_dupes` will **always** be `0` on the second call, which can lead to subtle bugs when using the function.\n",
    "\n",
    "This also illustrates imperative vs declarative style: the function now reads like a description of what we need, rather than a set of instructions to build up what we need.\n",
    "\n",
    "### Prefer simple argument and return types\n",
    "\n",
    "Functions should operate on data, rather than on custom objects, wherever possible. Prefer simple argument types like `dict`, `set`, `tuple`, `list`, `int`, `float`, and `bool`. Upgrade from there to standard library types like `datetime`, `timedelta`, `array`, `Decimal`, and `Future`. Only upgrade to your own custom types when absolutely necessary.\n",
    "\n",
    "As a good rule of thumb for whether your function is simple enough, ask yourself whether its arguments and return values could always be JSON-serializable. It turns out, this rule of thumb matters more than you might think: JSON-serializability is often a prerequisite to make the functions usable in parallel computing contexts. But, for the purpose of this document, the main benefits are: readability, testability, and overall function simplicity.\n",
    "\n",
    "### Avoid \"traditional\" OOP\n",
    "\n",
    "In \"traditional OOP languages\" like Java and C++, code re-use is achieved through class hierarchies and polymorphism, or so those languages claim. In Python, though we have the ability to subclass and to do class-based polymorphism, in practice, these capabilities are used rarely in idiomatic Python programs.\n",
    "\n",
    "It's more common to achieve re-use through modules and functions, and it's more common to achieve dynamic dispatch through duck typing. If you find yourself using super classes as a form of code re-use, stop what you're doing and reconsider. If you find yourself using lots of polymorphism, consider whether one of Python's dunder protocols or duck typing strategies might apply better.\n",
    "\n",
    "See also the excellent Python talk, [\"Stop Writing Classes\"][stop-classes], by a Python core contributor. In it, the presenter suggests that if you have built a class with a single method that is named like a class (e.g. `Runnable.run()`), then what you've done is modeled a function as a class, and you should just stop. Since in Python, functions are \"first-class\", there is **no reason** to do this!\n",
    "\n",
    "[stop-classes]: https://www.youtube.com/watch?v=o9pEzgHorH0\n",
    "\n",
    "### Mixins are sometimes OK\n",
    "\n",
    "One way to do class-based re-use without going overboard on type hierarchies is to use Mixins. Don't overuse these, though. \"Flat is better than nested\" applies to type hierarchies, too, so you should avoid introducing needless required layers of hierarchy just to decompose behavior.\n",
    "\n",
    "Mixins are not actually a Python language feature, but are possible thanks to its support for multiple inheritance. You can create base classes that \"inject\" functionality into your subclass without forming an \"important\" part of a type hierarchy, simply by listing that base class as the first entry in the `bases` list. An example:\n",
    "\n",
    "```python\n",
    "class APIHandler(AuthMixin, RequestHandler):\n",
    "    \"\"\"Handle HTTP/JSON requests with security.\"\"\"\n",
    "```\n",
    "\n",
    "The order matters, so may as well remember the rule: `bases` forms a hierarchy bottom-to-top. One readability benefit here is that everything you need to know about this class is contained in the `class` definition itself: \"it mixes in auth behavior and is a specialized Tornado RequestHandler.\"\n",
    "\n",
    "### Be careful with frameworks\n",
    "\n",
    "Python has a slew of frameworks for web, databases, and more. One of the joys of the language is that it's easy to create your own frameworks. When using an open source framework, you should be careful not to couple your \"core code\" too closely to the framework itself.\n",
    "\n",
    "When considering building your own framework for your code, you should err on the side of caution. The standard library has a lot of stuff built-in, PyPI has even more, and usually, [YAGNI applies][yagni].\n",
    "\n",
    "[yagni]: http://c2.com/cgi/wiki?YouArentGonnaNeedIt\n",
    "\n",
    "### Respect metaprogramming\n",
    "\n",
    "Python supports \"metaprogramming\" via a number of features, including decorators, context managers, descriptors, import hooks, metaclasses and AST transformations.\n",
    "\n",
    "You should feel comfortable using and understanding these features -- they are a core part of the language and are fully supported by it. But you should realize that when you use these features, you are opening yourself up to complex failure scenarios. Thus, treat the creation of metaprogramming facilities for your code similarly to the decision to \"build your own framework\". They amount to the same thing. When and if you do it, make the facilities into their own modules and document them well!\n",
    "\n",
    "### Don't be afraid of \"dunder\" methods\n",
    "\n",
    "Many people conflate Python's metaprogramming facilities with its support for \"double-underscore\" or \"dunder\" methods, such as `__getattr__`.\n",
    "\n",
    "As described in  the blog post, [\"Python double-under, double-wonder\"][dunder], there is nothing \"special\" about dunders. They are nothing more than a lightweight namespace the Python core developers picked for all of Python's internal protocols. After all, `__init__` is a dunder, and there's nothing magic about it.\n",
    "\n",
    "It's true that some dunders can create more confusing results than others -- for example, it's probably not a good idea to overload operators without good reason. But many of them, such as `__repr__`,  `__str__`, `__len__`, and `__call__` are really full parts of the language you should be leveraging in idiomatic Python code. Don't shy away!\n",
    "\n",
    "[dunder]: http://www.pixelmonkey.org/2013/04/11/python-double-under-double-wonder\n",
    "\n",
    "## A Little Zen for Your Code Style\n",
    "\n",
    "Barry Warsaw, one of the core Python developers, once said that it frustrated him that \"The Zen of Python\" ([PEP 20][pep20]) is used as a style guide for Python code, since it was originally written as a poem about Python's **internal** design. That is, the design of the language and language implementation itself. One can acknowledge that, but a few of the lines from PEP 20 serve as pretty good guidelines for idiomatic Python code, so we'll just go with it.\n",
    "\n",
    "[pep20]:  https://www.python.org/dev/peps/pep-0020/\n",
    "[pocoo-naming]: http://www.pocoo.org/internal/styleguide/#naming-conventions\n",
    "\n",
    "### Beautiful is better than ugly\n",
    "\n",
    "This one is subjective, but what it usually amounts to is this: will the person who inherits this code from you be impressed or disappointed? What if that person is you, three years later?\n",
    "\n",
    "### Explicit is better than implicit\n",
    "\n",
    "Sometimes in the name of refactoring out repetition in our code, we also get a little bit abstract with it. It should be possible to translate the code into plain English and basically understand what's going on. There shouldn't be an excessive amount of \"magic\".\n",
    "\n",
    "### Flat is better than nested\n",
    "\n",
    "This one is really easy to understand. The best functions have no nesting, neither by loops nor `if` statements. Second best is one level of nesting. Two or more levels of nesting, and you should probably start refactoring to smaller functions.\n",
    "\n",
    "Also, don't be afraid to refactor a nested if statement into a multi-part boolean conditional. For example:\n",
    "\n",
    "```python\n",
    "# bad\n",
    "if response:\n",
    "    if response.get(\"data\"):\n",
    "        return len(response[\"data\"])\n",
    "```\n",
    "\n",
    "is better written as:\n",
    "\n",
    "```python\n",
    "# good\n",
    "if response and response.get(\"data\"):\n",
    "    return len(response[\"data\"])\n",
    "```\n",
    "\n",
    "### Readability counts\n",
    "\n",
    "Don't be afraid to add line-comments with `#`. Don't go overboard on these or over-document, but a little explanation, line-by-line, often helps a whole lot. Don't be afraid to pick a slightly longer name because it's more descriptive. No one wins any points for shortening \"`response`\" to \"`rsp`\". Use doctest-style examples to illustrate edge cases in docstrings. Keep it simple!\n",
    "\n",
    "### Errors should never pass silently\n",
    "\n",
    "The biggest offender here is the bare `except: pass` clause. Never use these. Suppressing **all** exceptions is simply dangerous. Scope your exception handling to single lines of code, and always scope your `except` handler to a specific type. Also, get comfortable with the `logging` module and `log.exception(...)`.\n",
    "\n",
    "### If the implementation is hard to explain, it's a bad idea\n",
    "\n",
    "This is a general software engineering principle -- but applies very well to Python code. Most Python functions and objects can have an easy-to-explain implementation. If it's hard to explain, it's probably a bad idea. Usually you can make a hard-to-explain function easier-to-explain via \"divide and conquer\" -- split it into several functions.\n",
    "\n",
    "### Testing is one honking great idea\n",
    "\n",
    "OK, we took liberty on this one -- in \"The Zen of Python\", it's actually \"namespaces\" that's the honking great idea.\n",
    "\n",
    "But seriously: beautiful code without tests is simply worse than even the ugliest tested code. At least the ugly code can be refactored to be beautiful, but the beautiful code can't be refactored to be verifiably correct, at least not without writing the tests! So, write tests! Please!\n",
    "\n",
    "## Six of One, Half a Dozen of the Other\n",
    "\n",
    "This is a section for arguments we'd rather not settle. Don't rewrite other people's code because of this stuff. Feel free to use these forms interchangeably.\n",
    "\n",
    "### `str.format` vs overloaded format `%`\n",
    "\n",
    "`str.format` is more robust, yet `%` with `\"%s %s\"` printf-style strings is more concise. Both will be around forever.\n",
    "\n",
    "Remember to use unicode strings for your format pattern, if you need to preserve unicode:\n",
    "\n",
    "```python\n",
    "u\"%s %s\" % (dt.datetime.utcnow().isoformat(), line)\n",
    "```\n",
    "\n",
    "If you do end up using `%`, you should consider the `\"%(name)s\"` syntax which allows you to use a dictionary rather than a tuple, e.g.\n",
    "\n",
    "```python\n",
    "u\"%(time)s %(line)s\" % {\"time\": dt.datetime.utcnow().isoformat(), \"line\": line}\n",
    "```\n",
    "\n",
    "Also, don't re-invent the wheel. One thing `str.format` does unequivocally better is support various [formatting modes][str-format], such as humanized numbers and percentages. Use them.\n",
    "\n",
    "But use whichever one you please. We choose not to care.\n",
    "\n",
    "[str-format]: https://docs.python.org/2/library/string.html#formatspec\n",
    "\n",
    "### `if item` vs `if item is not None`\n",
    "\n",
    "This is unrelated to the earlier rule on `==` vs `is` for `None`. In this case, we are actually taking advantage of Python's \"truthiness rules\" to our benefit in `if item`, e.g. as a shorthand \"item is not None or empty string.\"\n",
    "\n",
    "Truthiness is a [tad complicated][truth-values] in Python and certainly the latter is safer against some classes of bugs. The former, however, is very common in much Python code, and it's shorter. We choose not to care.\n",
    "\n",
    "[truth-values]: https://docs.python.org/2/library/stdtypes.html#truth-value-testing\n",
    "\n",
    "### Implicit multi-line strings vs triple-quote `\"\"\"`\n",
    "\n",
    "Python's compiler will automatically join multiple quoted strings together into a single string during the parse phase if it finds nothing in between them, e.g.\n",
    "\n",
    "```python\n",
    "msg = (\"Hello, wayward traveler!\\n\"\n",
    "       \"What shall we do today?\\n\"\n",
    "       \"=>\")\n",
    "print(msg)\n",
    "```\n",
    "\n",
    "This is roughly equivalent to:\n",
    "\n",
    "```python\n",
    "msg = \"\"\"Hello, wayward traveler!\n",
    "What shall we do today?\n",
    "=>\"\"\"\n",
    "print(msg)\n",
    "```\n",
    "\n",
    "In the former's case, you keep the indentation clean, but need the ugly newline characters. In the latter case, you don't need the newlines, but break indentation. We choose not to care.\n",
    "\n",
    "### Using `raise` with classes vs instances\n",
    "\n",
    "It turns out Python lets you pass either an exception **class** or an exception **instance** to the `raise` statement. For example, these two lines are roughly equivalent:\n",
    "\n",
    "```python\n",
    "raise ValueError\n",
    "raise ValueError()\n",
    "```\n",
    "\n",
    "Essentially, Python turns the [first line into the second automatically][raises]. You should probably prefer the second form, if for no other reason than to **actually provide a useful argument**, like a helpful message about why the `ValueError` occurred. But these two lines **are** equivalent and you shouldn't rewrite one style into the other just because. We choose not to care.\n",
    "\n",
    "[raises]: https://docs.python.org/2.7/reference/simple_stmts.html#the-raise-statement\n",
    "\n",
    "## Standard Tools and Project Structure\n",
    "\n",
    "We've made some choices on \"best-of-breed\" tools for things, as well as the very minimal starting structure for a proper Python project.\n",
    "\n",
    "### The Standard Library\n",
    "\n",
    "- `import datetime as dt`: always import `datetime` this way\n",
    "- `dt.datetime.utcnow()`: preferred to `.now()`, which does local time\n",
    "- `import json`: the standard for data interchange\n",
    "- `from collections import namedtuple`: use for lightweight data types\n",
    "- `from collections import defaultdict`: use for counting/grouping\n",
    "- `from collections import deque`: a fast double-ended queue\n",
    "- `from itertools import groupby, chain`: for declarative style\n",
    "- `from functools import wraps`: use for writing well-behaved decorators\n",
    "- `argparse`: for \"robust\" CLI tool building\n",
    "- `fileinput`: to create quick UNIX pipe-friendly tools\n",
    "- `log = logging.getLogger(__name__)`: good enough for logging\n",
    "- `from __future__ import absolute_import`: fixes import aliasing\n",
    "\n",
    "### Common Third-Party Libraries\n",
    "\n",
    "- `python-dateutil` for datetime parsing and calendars\n",
    "- `pytz` for timezone handling\n",
    "- `tldextract` for better URL handling\n",
    "- `msgpack-python` for a more compact encoding than JSON\n",
    "- `futures` for Future/pool concurrency primitives\n",
    "- `docopt` for quick throwaway CLI tools\n",
    "- `py.test` for unit tests, along with `mock` and `hypothesis`\n",
    "\n",
    "### Local Development Project Skeleton\n",
    "\n",
    "For all Python packages and libraries:\n",
    "\n",
    "- no `__init__.py` in root folder: give your package a folder name!\n",
    "- `mypackage/__init__.py` preferred to `src/mypackage/__init__.py`\n",
    "- `mypackage/lib/__init__.py` preferred to `lib/__init__.py`\n",
    "- `mypackage/settings.py` preferred to `settings.py`\n",
    "- `README.rst` describes the repo for a newcomer; use reST\n",
    "- `setup.py` for simple facilities like `setup.py develop`\n",
    "- `requirements.txt` describes package dependencies for `pip`\n",
    "- `dev-requirements.txt` additional dependencies for tests/local\n",
    "- `Makefile` for simple (!!!) build/lint/test/run steps\n",
    "\n",
    "Also, always [pin your requirements](http://nvie.com/posts/better-package-management/).\n",
    "\n",
    "## Some Inspiration\n",
    "\n",
    "The following links may give you some inspiration about the core of writing Python code with great style and taste.\n",
    "\n",
    "- Python's stdlib [`Counter` class][Counter], implemented by Raymond Hettinger\n",
    "- The [`rq.queue` module][rq], originally by Vincent Driessen\n",
    "- This document's author also wrote [this blog post on \"Pythonic\" code][idiomatic]\n",
    "\n",
    "Go forth and be Pythonic!\n",
    "\n",
    "```\n",
    "$ python\n",
    ">>> import antigravity\n",
    "```\n",
    "\n",
    "[Counter]: https://github.com/python/cpython/blob/57b569d8af2b3263c5d9e6d75fb308f89ea17ac6/Lib/collections/__init__.py#L446-L841\n",
    "[rq]: https://github.com/nvie/rq/blob/master/rq/queue.py\n",
    "[idiomatic]: http://www.pixelmonkey.org/2010/11/03/pythonic-means-idiomatic-and-tasteful\n",
    "\n",
    "## Contributors\n",
    "\n",
    "- Andrew Montalenti ([@amontalenti][amontalenti]): original author\n",
    "- Vincent Driessen ([@nvie][nvie]): edits and suggestions\n",
    "\n",
    "[amontalenti]: http://twitter.com/amontalenti\n",
    "[nvie]: http://twitter.com/nvie\n",
    "\n",
    "---\n",
    "\n",
    "Like good Python style? Then perhaps you'd like to [work on our team of Pythonistas][tweet] at Parse.ly!\n",
    "\n",
    "[tweet]: https://twitter.com/amontalenti/status/682968375702716416"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}