Release notes and cleanup.

Author: waylan

docs/change_log/release-3.3.md

@@ -81,6 +81,13 @@ The following new features have been included in the 3.3 release:
   maintain the current behavior in the rebuilt Markdown in HTML extension. A few random
   edge-case bugs (see the included tests) were resolved in the process (#803).
 
+* An alternate method `markdown.extensions.headerid.slugify_unicode` has been included
+  with the [Table of Contents](../extensions/toc.md) extension which supports Unicode
+  characters in table fo contents slugs. The old `markdown.extensions.headerid.slugify`
+  method which removes non-ASCII characters remains the default. Import and pass
+  `markdown.extensions.headerid.slugify_unicode` to the `slugify` configuration option
+  to use the new behavior.
+
 ## Bug fixes
 
 The following bug fixes are included in the 3.3 release:

docs/extensions/toc.md

@@ -202,7 +202,8 @@ The following options are provided to configure the output:
 
     The callable must return a string appropriate for use in HTML `id` attributes.
 
-    An alternate version of the default callable supporting Unicode strings is also provided as `markdown.extensions.headerid.slugify_unicode`.
+    An alternate version of the default callable supporting Unicode strings is also
+    provided as `markdown.extensions.headerid.slugify_unicode`.
 
 * **`separator`**:
     Word separator. Character which replaces white space in id. Defaults to "`-`".

markdown/extensions/toc.py

@@ -23,20 +23,16 @@
 import xml.etree.ElementTree as etree
 
 
-def _slugify(value, separator, encoding='ascii'):
+def slugify(value, separator, encoding='ascii'):
+    """ Slugify a string, to make it URL friendly. """
     value = unicodedata.normalize('NFKD', value).encode(encoding, 'ignore')
     value = re.sub(r'[^\w\s-]', '', value.decode(encoding)).strip().lower()
-    return re.sub(r'[%s\s]+' % separator, separator, value)
-
-
-def slugify(value, separator):
-    """ Slugify a string, to make it URL friendly. """
-    return _slugify(value, separator)
+    return re.sub(r'[{}\s]+'.format(separator), separator, value)
 
 
 def slugify_unicode(value, separator):
-    """ Slugify a string, to make it URL friendly. """
-    return _slugify(value, separator, 'utf-8')
+    """ Slugify a string, to make it URL friendly while preserving Unicode characters. """
+    return slugify(value, separator, 'utf-8')
 
 
 IDCOUNT_RE = re.compile(r'^(.*)_([0-9]+)$')