09_Useful_Packages.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Useful Packages\n",
    "\n",
    "The number of registered packages in the [General](https://github.com/JuliaRegistries/General/blob/master/Registry.toml) registry is quite vast (about 7100 on **16/02/2022**) and growing every day. So if you have a reasonably standard problem you want to solve, there is a good chance that someone has already made an effort. Even if that's not the case the composability between packages enabled by the Julia language implies that often building blocks from other packages can be easily reused as well. There is also a [great search engine](https://juliahub.com/ui/Packages) for Julia packages.\n",
    "\n",
    "To give you some ideas about the community and what is available, here is a non-exhaustive list, which is naturally influenced by my own background. I'm happy to adapt it, if you feel something is wrong / missing.\n",
    "\n",
    "Without further ado:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Development and coding workflow\n",
    "\n",
    "See also the [workflow tips](https://docs.julialang.org/en/v1/manual/workflow-tips/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [Revise](https://github.com/timholy/Revise.jl) is one of the absolute must-have packages. It watches your filesystem for changes to a Julia package or script and reloads it automatically in the current REPL / Jupyter kernel. If you are the developer of a Julia package or usually find yourself coding in one terminal and using the code in another, this package is indispensable.\n",
    "- [JuliaFormatter](https://github.com/domluna/JuliaFormatter.jl): Basic automatic source-code formatter for Julia.\n",
    "- [Infiltrator](https://github.com/JuliaDebug/Infiltrator.jl): Drop into an interactive REPL at an arbitrary place in your code. Great for debugging and testing.\n",
    "- [Debugger](https://github.com/JuliaDebug/Debugger.jl): A gdb-like Julia debugger, which I use every day.\n",
    "- [Rebugger](https://github.com/timholy/Rebugger.jl): Another debugger. Preferred over Debugger by some people.\n",
    "- [Julia-VSCode](https://www.julia-vscode.org/): Not actually a package, but the development environment endorsed by Julia computing.\n",
    "- Surely there is Julia support for [your favourite editor](https://github.com/JuliaEditorSupport), too."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reports and source-code documentation\n",
    "- [Documenter](https://github.com/JuliaDocs/Documenter.jl) is the goto source code documentation tool for Julia.\n",
    "- [Literate](https://github.com/fredrikekre/Literate.jl/) allows to mix markdown and Julia, where the Julia code is automatically executed and resulting plots, data, tables are included into the final document. Supports generation of HTML files or Jupyter notebooks as well.\n",
    "- [IJulia](https://github.com/JuliaLang/IJulia.jl): Integration of Julia with Jupyter notebooks\n",
    "- [Pluto](https://github.com/fonsp/Pluto.jl): Interactive reactive notebooks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More linear algebra\n",
    "\n",
    "- [IterativeSolvers](https://juliamath.github.io/IterativeSolvers.jl/dev/) is a rich library for iterative methods with the most important iterative solver algorithms (Conjugate Gradient, GMRES, BiCGStab, Chebychev iteration, ...)\n",
    "- [StaticArrays](https://github.com/JuliaArrays/StaticArrays.jl): Arrays of compile-time known size. Has a huge speed improvement for example when used for 3D vectors or 3D rotations.\n",
    "- [BlockArrays](https://github.com/JuliaArrays/BlockArrays.jl): Special array for block-wise storing or treating arrays \n",
    "- [LazyArrays](https://github.com/JuliaArrays/LazyArrays.jl): Makes array operations lazy.\n",
    "- [Tullio](https://github.com/mcabbott/Tullio.jl/): Extremely flexible package for higher-order tensor operations in an intuitive syntax. Support for GPU and AD.\n",
    "- [TensorOperations](https://github.com/Jutho/TensorOperations.jl): Similar to Tullio with a slightly varying feature set. Sometimes one, sometimes the other is better."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Automatic Differentiation (AD)\n",
    "- [Zygote](https://github.com/FluxML/Zygote.jl): Adjoint-mode AD\n",
    "- [ForwardDiff](https://github.com/JuliaDiff/ForwardDiff.jl): Forward-mode AD\n",
    "- [FiniteDifferences](https://github.com/JuliaDiff/FiniteDifferences.jl): Classic finite-difference approximations\n",
    "- [ChainRules](https://github.com/JuliaDiff/ChainRules.jl): Generic interface for automatice differentiation in Julia, allows to couple different kinds of AD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualisation and interaction \n",
    "- [Plots](https://github.com/JuliaPlots/Plots.jl) (Generic plotting interface metapackage)\n",
    "- [PyPlot](https://github.com/JuliaPy/PyPlot.jl) (Plotting via python and matplotlib)\n",
    "[PGFPlotsX](https://github.com/KristofferC/PGFPlotsX.jl) (Plotting via TeX, TikZ and pgf)\n",
    "- [Gladify](https://github.com/GiovineItalia/Gadfly.jl) (Pure Julia plotting implementation, widespread)\n",
    "- [PlotlyJS](https://github.com/sglyon/PlotlyJS.jl) (Ploting via Javascript and Plotly)\n",
    "- [GR](https://github.com/jheinen/GR.jl) (Plotting based on the GR framework, pretty fast)\n",
    "- [Makie](https://github.com/JuliaPlots/Makie.jl) (Extremely feature-rich; can plot on the GPU directly)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Symbolic manipulation\n",
    "- [SymPy](https://github.com/JuliaPy/SymPy.jl): Julia wrapper of the popular python package\n",
    "- [Symbolics](https://juliasymbolics.org/): Pure-Julia Computer Algebra System (CAS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standard file formats\n",
    "\n",
    "Plenty of standard file formats in scientific computing and elsewhere can be used in Julia:\n",
    "- [JSON](https://github.com/JuliaIO/JSON.jl) and [YAML](https://github.com/BioJulia/YAML.jl) (TOML is in the standard library)\n",
    "- [HDF5](https://github.com/JuliaIO/HDF5.jl)\n",
    "- [MAT](https://github.com/JuliaIO/MAT.jl): Support for Matlab's mat files\n",
    "- [NCDatasets](https://github.com/Alexander-Barth/NCDatasets.jl): Support for netcdf\n",
    "- [JLD](https://github.com/JuliaIO/JLD.jl) and [JLD2](https://github.com/JuliaIO/JLD2.jl): Julia's own file format to reproducably dump and load any Julia object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chemistry and Materials Science\n",
    "\n",
    "\n",
    "- [DFTK](https://github.com/JuliaMolSim/DFTK.jl): Toolkit for density-functional theory and related models in extended systems\n",
    "- [Fermi](https://github.com/FermiQC/Fermi.jl): Small toolkit for molecular Post-HF calculations using Gaussian basis sets.\n",
    "- [AtomsBase](https://github.com/JuliaMolSim/AtomsBase.jl): Interfacing framework for molecular and periodic structures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PDEs and ODEs\n",
    "- Most of the tooling for solving differential equations in Julia is collected in the [JuliaDiffEq](https://juliadiffeq.org/) organisation. The most important package is [DifferentialEquations](https://github.com/JuliaDiffEq/DifferentialEquations.jl), where basically all the strings are collected. The package mostly concentrates on ordinary differential equations in various kinnds (random, stochastic, algebraic, mixed discrete / continuous ...)\n",
    "- For PDEs there are a couple of emerging implementations. We already briefly talked about [Gridap](https://github.com/gridap/Gridap.jl). But there also is [Ferrite](https://github.com/Ferrite-FEM/Ferrite.jl), which seems to have a more hackable design."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimisation-related packages\n",
    "\n",
    "The optimisation community has adopted Julia rather early and the respective Julia libraries are at a very good shape.\n",
    "\n",
    "- The driving force behind the Julia optimisation community is the [JuliaOpt](https://www.juliaopt.org/) organisation with its working horse [JuMP](https://github.com/JuliaOpt/JuMP.jl). This package defines a metalanguage for optimisation problems, which can be combined with about 20 open-source or commercial optimisation solvers. A pretty exhaustive set of problem classes are supported: Linear programming, (mixed) integer programming, semidefinite programming, nonlinear programming, ...\n",
    "\n",
    "- For certain highly specific use cases a number of specialised packages have emerged:\n",
    "    - [Optim](https://github.com/JuliaNLSolvers/Optim.jl) for continuous optimisation problems including simple manifolds\n",
    "    - [LsqFit](https://github.com/JuliaNLSolvers/LsqFit.jl) for least-squares problems\n",
    "    - [Roots](https://github.com/JuliaMath/Roots.jl) for finding roots of functions\n",
    "    - [NLsolve](https://github.com/JuliaNLSolvers/NLsolve.jl) for general non-linear systems of equations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data science and statistics\n",
    "Apart from a few examples, the most important statistics and data science packages are associated with the [JuliaStats](https://github.com/JuliaStats) and [JuliaData](https://github.com/JuliaData) organisations. In particular there are:\n",
    "- `Statistics` in the standard library.\n",
    "- [DataFrames](https://github.com/JuliaData/DataFrames.jl): Package for working with tabular data (\"the pandas of Julia\")\n",
    "- [Queryverse](http://www.queryverse.org/): Metapackage for datascience and large-scale queries\n",
    "- [PrettyTables](https://github.com/ronisbr/PrettyTables.jl): Nicely formatted tables\n",
    "- [StatsKit](https://github.com/JuliaStats/StatsKit.jl): Metapackage loading a full suite of Statistics-related tools\n",
    "- [Distributions](https://github.com/JuliaStats/Distributions.jl): Defines standard distributions and their properties\n",
    "- [Turing](https://github.com/TuringLang/Turing.jl): Bayesian inference and probabilistic programming\n",
    "- [BayesianOptimization](https://github.com/jbrea/BayesianOptimization.jl)\n",
    "- [GaussianProcesses](https://github.com/STOR-i/GaussianProcesses.jl): Fit and work with Gaussian Processes\n",
    "- [Stheno](https://github.com/JuliaGaussianProcesses/Stheno.jl): Another Gaussian process package\n",
    "- [BAT.jl](https://github.com/bat/BAT.jl): Bayesian Analysis Toolkit (MCMC sampling, ...)\n",
    "- [BayesianLinearRegressors](https://github.com/JuliaGaussianProcesses/BayesianLinearRegressors.jl): Bayesian Linear Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Machine Learning\n",
    "\n",
    "The most important machine-learning framework in Julia is [Flux](https://fluxml.ai/). It's ecosystem features packages such as:\n",
    "- [Flux.jl](https://github.com/FluxML/Flux.jl): The Flux backend\n",
    "- [FluxTraining.jl](https://github.com/FluxML/FluxTraining.jl): Tools for setting up and training neural nets.\n",
    "- See also the [Flux ecosystem](https://fluxml.ai/ecosystem.html) overview"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": "0bba521938544a5096c2349d28e65280",
   "lastKernelId": "e13bd870-ac07-446d-a160-212f94b7a565"
  },
  "kernelspec": {
   "display_name": "Julia 1.7.2",
   "language": "julia",
   "name": "julia-1.7"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}