diff --git a/.gitignore b/.gitignore index 36f2cc5..32e1792 100644 --- a/.gitignore +++ b/.gitignore @@ -2,4 +2,5 @@ *.DS_Store *~ django_debug_logging.egg-info -dist \ No newline at end of file +dist +_build diff --git a/README.rst b/README.rst index aa2364d..66e3064 100644 --- a/README.rst +++ b/README.rst @@ -49,159 +49,12 @@ A log record: :alt: Debug Logging detail view :target: https://github.com/lincolnloop/django-debug-logging/raw/develop/docs/screenshots/debug_logging_3.png -Prerequisites -------------- - -These requirements are installed automatically by *pip* and *easy_install*, and -are in the included *requirements.pip* file. - -`Django Debug Toolbar`_ - This project is designed to work alongside the Django -Debug Toolbar and extend its functionality to support logging. - -Picklefield_ - Used to saved pickled versions of settings, sql queries, and -cache calls to the database. - -Installation ------------- - -Before you begin, make sure Django Debug Toolbar is configured and working -properly. - -Install the project with pip:: - - $ pip install django-debug-logging - -This should install django-picklefield as well, which is needed. - -Next, you'll add *debug_logging* to your INSTALLED_APPS:: - - INSTALLED_APPS = ( - ... - 'debug_toolbar', - 'debug_logging', - ) - -Now, you'll need to replace the standard DebugToolbarMiddleware with a -middleware that extends it to add logging functionality. The toolbar will -still function normally when logging is disabled. - -From your MIDDLEWARE_CLASSES setting, remove:: - - 'debug_toolbar.middleware.DebugToolbarMiddleware', - -Replace it with:: - - 'debug_logging.middleware.DebugLoggingMiddleware', - -Now, you'll need to replace a few of the panels with extended versions that -support logging. If you don't want the data from any one of these panels to -be logged, you can skip it. - -From your DEBUG_TOOLBAR_PANELS setting, remove:: - - 'debug_toolbar.panels.cache.CacheDebugPanel', - 'debug_toolbar.panels.settings_vars.SettingsVarsDebugPanel', - 'debug_toolbar.panels.sql.SQLDebugPanel', - 'debug_toolbar.panels.timer.TimerDebugPanel', - -Replace them with:: - - 'debug_logging.panels.cache.CacheLoggingPanel', - 'debug_logging.panels.settings_vars.SettingsVarsLoggingPanel', - 'debug_logging.panels.sql.SQLLoggingPanel', - 'debug_logging.panels.timer.TimerLoggingPanel', - -There are also a couple of panels that are unique to Django Debug Logging that -you may find convenient when logging data over time. If you'd like, you can -add them to your DEBUG_TOOLBAR_PANELS setting:: - - 'debug_logging.panels.revision.RevisionLoggingPanel', - 'debug_logging.panels.identity.IdentityLoggingPanel', - -Add the debug logging urls to your urls.py:: - - urlpatterns = patterns('', - ... - url(r'^debug-logging/', include('debug_logging.urls')), - ) - -The Debug Logger will ignore requests made to this frontend interface, so your -log won't be clogged with information you have no use for. - -Finally, run syncdb to create the models for statistic logging:: - - $ python manage.py syncdb - -South migrations are included in case migrations are needed when upgrading to -new versions. - -Requests are logged when they contain a 'DJANGO_DEBUG_LOGGING' header set to -True. This header is added automatically by the 'log_urls' command when it is -used. To prevent any performance impact from the rendering of the Debug Toolbar, it -is not shown when this header is present. - -For the best results, don't use the site while a test run is in progress. - -Settings --------- - -* ``SQL_EXTRA``: This setting determines whether the full details of each query - are logged, or just the number of queries and the total time. It defaults to - ``False``. - -* ``CACHE_EXTRA``: This determines whether the full details of each cache call - are logged, or just the summary details. It defaults to `` False``. - -* ``BLACKLIST``: Add a list of url prefixes that you would like to exclude from - logging here. The url for the Debug Logging frontend interface is added to - this blacklist automatically. - -Running a Url Test ------------------- - -A management command is included that uses the test client to hit a list of -urls in sequence, allowing them to be logged to the database. To use it, first -create a list of urls with a new url on each line. Lines beginning with # are -ignored. :: - - # Main urls - / - /my/url/ - /my/other/url/ - # Comments - /my/comment/url/ - -Then, enable logging and run the *log_urls* management command:: - - $ python manage.py log_urls myapp/my_urls.txt - -Unless it is run with a verbosity of 0 the command will output status -messages, such as urls that return codes other than 200 and urls that raise -errors. - -To run the test as an authenticated user, use the username and password -options:: - - $ python manage.py log_urls my_urls.txt --username Legen --password dary - -You can also add a name and a description to your run, if you'd like:: - - $ python manage.py log_urls my_urls.txt --name "Admin Urls" --description "Urls used by site admins" - -If you'd like to conduct a test run with a tool other than the log_urls -management command, you can use the command to manually start and end TestRun -objects, so that your results will be organized correctly in the UI. Before you -conduct your test, simply run:: - - $ python manage.py log_urls --manual-start - -Then, when you are finished hitting your desired urls:: - - $ python manage.py log_urls --manual-end - To Do ----- +We welcome contributions! Here are some of our main priorities for continued +development: + * Add a --repeat option to the log_urls command so that the urls can be run through multiple times. diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 0000000..81cd700 --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,130 @@ +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = sphinx-build +PAPER = +BUILDDIR = _build + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . + +.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest + +help: + @echo "Please use \`make ' where is one of" + @echo " html to make standalone HTML files" + @echo " dirhtml to make HTML files named index.html in directories" + @echo " singlehtml to make a single large HTML file" + @echo " pickle to make pickle files" + @echo " json to make JSON files" + @echo " htmlhelp to make HTML files and a HTML help project" + @echo " qthelp to make HTML files and a qthelp project" + @echo " devhelp to make HTML files and a Devhelp project" + @echo " epub to make an epub" + @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" + @echo " latexpdf to make LaTeX files and run them through pdflatex" + @echo " text to make text files" + @echo " man to make manual pages" + @echo " changes to make an overview of all changed/added/deprecated items" + @echo " linkcheck to check all external links for integrity" + @echo " doctest to run all doctests embedded in the documentation (if enabled)" + +clean: + -rm -rf $(BUILDDIR)/* + +html: + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." + +dirhtml: + $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." + +singlehtml: + $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml + @echo + @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." + +pickle: + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle + @echo + @echo "Build finished; now you can process the pickle files." + +json: + $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json + @echo + @echo "Build finished; now you can process the JSON files." + +htmlhelp: + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp + @echo + @echo "Build finished; now you can run HTML Help Workshop with the" \ + ".hhp project file in $(BUILDDIR)/htmlhelp." + +qthelp: + $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp + @echo + @echo "Build finished; now you can run "qcollectiongenerator" with the" \ + ".qhcp project file in $(BUILDDIR)/qthelp, like this:" + @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/django-debug-logging.qhcp" + @echo "To view the help file:" + @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/django-debug-logging.qhc" + +devhelp: + $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp + @echo + @echo "Build finished." + @echo "To view the help file:" + @echo "# mkdir -p $$HOME/.local/share/devhelp/django-debug-logging" + @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/django-debug-logging" + @echo "# devhelp" + +epub: + $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub + @echo + @echo "Build finished. The epub file is in $(BUILDDIR)/epub." + +latex: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo + @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." + @echo "Run \`make' in that directory to run these through (pdf)latex" \ + "(use \`make latexpdf' here to do that automatically)." + +latexpdf: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo "Running LaTeX files through pdflatex..." + $(MAKE) -C $(BUILDDIR)/latex all-pdf + @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." + +text: + $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text + @echo + @echo "Build finished. The text files are in $(BUILDDIR)/text." + +man: + $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man + @echo + @echo "Build finished. The manual pages are in $(BUILDDIR)/man." + +changes: + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes + @echo + @echo "The overview file is in $(BUILDDIR)/changes." + +linkcheck: + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck + @echo + @echo "Link check complete; look for any errors in the above output " \ + "or in $(BUILDDIR)/linkcheck/output.txt." + +doctest: + $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest + @echo "Testing of doctests in the sources finished, look at the " \ + "results in $(BUILDDIR)/doctest/output.txt." diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 0000000..e7c5f2e --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,221 @@ +# -*- coding: utf-8 -*- +# +# django-debug-logging documentation build configuration file, created by +# sphinx-quickstart on Wed Nov 30 09:06:54 2011. +# +# This file is execfile()d with the current directory set to its containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys, os + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +#sys.path.insert(0, os.path.abspath('.')) + +DOCS_BASE = os.path.dirname(__file__) +sys.path.insert(0, os.path.abspath(os.path.join(DOCS_BASE, '..'))) + +import debug_logging + +# -- General configuration ----------------------------------------------------- + +# If your documentation needs a minimal Sphinx version, state it here. +#needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be extensions +# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. +extensions = ['sphinx.ext.autodoc'] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The encoding of source files. +#source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'django-debug-logging' +copyright = u'2011, Lincoln Loop' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = debug_logging.get_version(short=True) +# The full version, including alpha/beta/rc tags. +release = debug_logging.__version__ + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +#language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +#today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = ['_build'] + +# The reST default role (used for this markup: `text`) to use for all documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +#modindex_common_prefix = [] + + +# -- Options for HTML output --------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = 'default' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +#html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +#html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +#html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +#html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +#html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +#html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +#html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +#html_domain_indices = True + +# If false, no index is generated. +#html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, links to the reST sources are added to the pages. +#html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +#html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +#html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = None + +# Output file base name for HTML help builder. +htmlhelp_basename = 'django-debug-loggingdoc' + + +# -- Options for LaTeX output -------------------------------------------------- + +# The paper size ('letter' or 'a4'). +#latex_paper_size = 'letter' + +# The font size ('10pt', '11pt' or '12pt'). +#latex_font_size = '10pt' + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, author, documentclass [howto/manual]). +latex_documents = [ + ('index', 'django-debug-logging.tex', u'django-debug-logging Documentation', + u'Lincoln Loop', 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# If true, show page references after internal links. +#latex_show_pagerefs = False + +# If true, show URL addresses after external links. +#latex_show_urls = False + +# Additional stuff for the LaTeX preamble. +#latex_preamble = '' + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +#latex_domain_indices = True + + +# -- Options for manual page output -------------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + ('index', 'django-debug-logging', u'django-debug-logging Documentation', + [u'Lincoln Loop'], 1) +] diff --git a/docs/index.rst b/docs/index.rst new file mode 100644 index 0000000..24bdaa4 --- /dev/null +++ b/docs/index.rst @@ -0,0 +1,42 @@ +.. django-debug-logging documentation master file, created by + sphinx-quickstart on Wed Nov 30 09:06:54 2011. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +Welcome to django-debug-logging's documentation! +================================================ + +Django Debug Logging is a "plugin" for the `Django Debug Toolbar`_ that allows +users to log the debug toolbar statistics to the database during a site crawl. +This allows users to create performance testing plans to exercise the site, and +then review and aggregate the results afterwards to identify performance +problems. + +It also provides a basic UI for browsing the details that have been logged to +the database and reviewing aggregated information about test runs. The UI +borrows a lot from the custom Sphinx theme by the Read the Docs team, and the +Sentry project from Disqus. + +The overall goal is to use this tool to monitor performance statistics over +time, so that you can see trends and spikes in the number of queries, cache +misses, cpu time, etc., and identify where in the app the problems are coming +from. It is not intended as a load testing tool, so features like concurrency +and warmup periods will not be part of the initial focus. + +.. _Django Debug Toolbar: https://github.com/django-debug-toolbar/django-debug-toolbar + +Contents: + +.. toctree:: + :maxdepth: 2 + + install + settings + running + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/docs/install.rst b/docs/install.rst new file mode 100644 index 0000000..f72f90b --- /dev/null +++ b/docs/install.rst @@ -0,0 +1,101 @@ +Installation +============ + +Prerequisites +------------- + +These requirements are installed automatically by *pip* and *easy_install*, and +are in the included *requirements.pip* file. + +`Django Debug Toolbar`_ - This project is designed to work alongside the Django +Debug Toolbar and extend its functionality to support logging. + +Picklefield_ - Used to saved pickled versions of settings, sql queries, and +cache calls to the database. + +.. _Django Debug Toolbar: https://github.com/django-debug-toolbar/django-debug-toolbar +.. _Picklefield: https://github.com/gintas/django-picklefield + +Installing +---------- + +Before you begin, make sure Django Debug Toolbar is configured and working +properly. + +Install the project with pip:: + + $ pip install django-debug-logging + +This should install django-picklefield as well, which is needed. + +Next, you'll add *debug_logging* to your INSTALLED_APPS:: + + INSTALLED_APPS = ( + ... + 'debug_toolbar', + 'debug_logging', + ) + +Now, you'll need to replace the standard DebugToolbarMiddleware with a +middleware that extends it to add logging functionality. The toolbar will +still function normally when logging is disabled. + +From your MIDDLEWARE_CLASSES setting, remove:: + + 'debug_toolbar.middleware.DebugToolbarMiddleware', + +Replace it with:: + + 'debug_logging.middleware.DebugLoggingMiddleware', + +Now, you'll need to replace a few of the panels with extended versions that +support logging. If you don't want the data from any one of these panels to +be logged, you can skip it. + +From your DEBUG_TOOLBAR_PANELS setting, remove:: + + 'debug_toolbar.panels.cache.CacheDebugPanel', + 'debug_toolbar.panels.settings_vars.SettingsVarsDebugPanel', + 'debug_toolbar.panels.sql.SQLDebugPanel', + 'debug_toolbar.panels.timer.TimerDebugPanel', + +Replace them with:: + + 'debug_logging.panels.cache.CacheLoggingPanel', + 'debug_logging.panels.settings_vars.SettingsVarsLoggingPanel', + 'debug_logging.panels.sql.SQLLoggingPanel', + 'debug_logging.panels.timer.TimerLoggingPanel', + +There are also a couple of panels that are unique to Django Debug Logging that +you may find convenient when logging data over time. If you'd like, you can +add them to your DEBUG_TOOLBAR_PANELS setting:: + + 'debug_logging.panels.revision.RevisionLoggingPanel', + 'debug_logging.panels.identity.IdentityLoggingPanel', + +Add the debug logging urls to your urls.py:: + + urlpatterns = patterns('', + ... + url(r'^debug-logging/', include('debug_logging.urls')), + ) + +The Debug Logger will ignore requests made to this frontend interface, so your +log won't be clogged with information you have no use for. + +Finally, run syncdb to create the models for statistic logging:: + + $ python manage.py syncdb + +South migrations are included in case migrations are needed when upgrading to +new versions. + +Logging +------- + +Requests are logged when they contain a 'DJANGO_DEBUG_LOGGING' header set to +True. This header is added automatically by the 'log_urls' command when it is +used. To prevent any performance impact from the rendering of the Debug Toolbar, +it is not shown when this header is present. + +For the best results, don't use the site while a test run is in progress. diff --git a/docs/running.rst b/docs/running.rst new file mode 100644 index 0000000..816bc68 --- /dev/null +++ b/docs/running.rst @@ -0,0 +1,42 @@ +Running a Url Test +================== + +A management command is included that uses the test client to hit a list of +urls in sequence, allowing them to be logged to the database. To use it, first +create a list of urls with a new url on each line. Lines beginning with # are +ignored. :: + + # Main urls + / + /my/url/ + /my/other/url/ + # Comments + /my/comment/url/ + +Then, enable logging and run the *log_urls* management command:: + + $ python manage.py log_urls myapp/my_urls.txt + +Unless it is run with a verbosity of 0 the command will output status +messages, such as urls that return codes other than 200 and urls that raise +errors. + +To run the test as an authenticated user, use the username and password +options:: + + $ python manage.py log_urls my_urls.txt --username Legen --password dary + +You can also add a name and a description to your run, if you'd like:: + + $ python manage.py log_urls my_urls.txt --name "Admin Urls" --description "Urls used by site admins" + +If you'd like to conduct a test run with a tool other than the log_urls +management command, you can use the command to manually start and end TestRun +objects, so that your results will be organized correctly in the UI. Before you +conduct your test, simply run:: + + $ python manage.py log_urls --manual-start + +Then, when you are finished hitting your desired urls:: + + $ python manage.py log_urls --manual-end diff --git a/docs/settings.rst b/docs/settings.rst new file mode 100644 index 0000000..f13acfe --- /dev/null +++ b/docs/settings.rst @@ -0,0 +1,13 @@ +Settings +======== + +* ``SQL_EXTRA``: This setting determines whether the full details of each query + are logged, or just the number of queries and the total time. It defaults to + ``False``. + +* ``CACHE_EXTRA``: This determines whether the full details of each cache call + are logged, or just the summary details. It defaults to `` False``. + +* ``BLACKLIST``: Add a list of url prefixes that you would like to exclude from + logging here. The url for the Debug Logging frontend interface is added to + this blacklist automatically.