This repo exists as part curriculum and part textbook for Jonathan's art and machine learning virtual "course". The course is mainly an indefinite Twitch series in the form of live lectures/workshops. Recordings of the streams will be edited and posted to YouTube as VODs. Lecture notes and exercises will form an informal interactive textbook and reference if reading is more your speed.
These materials are a top-down application-first appproach to both understanding how to use deep learning for art as well as an exploration of neural networks as art (this will make sense later in the course). All code examples will be in Javascript using Tensorflow.js.
Whether you are an artist or designer looking to augment your practice with machine learning or a ML engineer/researcher wanting to make creative applications there should be something for you here!
- Prototyping a model in a DL reseach paper.
- Understanding and staying up to date on recent advances.
- Critically critiquing art created with AI.
- Creating you own generative art piece (and possibly minting it as a NFT).
Feel free to engage with as many (or as few) of these resources as you want.
| Live | Notes/Exercises/Examples | Recordings | Chat |
|---|---|---|---|
| Twitch | This repo! | YouTube | Discord |
Art and Machine Learning resources and references (with some new media thrown in).
- Machine Learning for Artists
- ITP @ NYU: Machine Learning for the Web
- ITP @ NYU: Introduction to Computational Media
- Kadenze: Creative Applications of Deep Learning with TensorFlow
- Kadenze: Generative Art and Computational Creativity
- CMU: Art and Machine Learning
- CMU: New Media Installation: Art that Learns
- Coding Train: Youtube Playlists
Here is a list of some common creative coding libraries (not including machine learning specific libraries).
- Processing (Java, p5.js, and processing.py)
- HTML/Javascript (canvas and two.js)
- 3D/WebGL (three.js, regl, stack.gl, Mathbox)
- Interactive & Installation (openFrameworks and Cinder)
- Audio (tone.js, Max, Pure Data, Overtone, ChucK, Sonic Pi)
- Text (spacy, natural, Tracery)
Not sure where to start/what to use, try p5.js or processing.py 🙌
- PyTorch
- Tensorflow
- Keras
- Darknet
- dlib
- CNTK (with C# interface)
- scikit-learn
- Unity ML-Agents
- Various existing GANs (vegans)
- https://codepen.io: My favorite for low overhead/commitment rapid client side experimentation. Split screen views, live reload, sharable, no account necessary, etc.
- https://glitch.com: My all time favorite for full app/website experimentation and prototyping (or even hosting). Feature rich editor with multi-file and console support. Like a computer in the browser. Does serverside Nodejs and gives you a full virtual machine in the browser environment. Really great community, built for remixing/sharing, easy to go from prototype to production-y app effortlessly. Learner/beginner friendly.
- https://observablehq.com: Reactive notebook environment (from creators of D3, Backbone.js, etc.). Has some very powerful features but the environment is slighly idiosyncratic (different than normal coding in an editor). Really well suited for remixing, prototyping, and sequential things.
- https://codesandbox.io/: IDE in the browser. targeted at rich client side applications (React, Vue, Svelte, etc.) and more development/production-y things.
- https://repl.it/: Multi language support.
- https://runkit.com: Similar notebook-like environment. targeted at Nodejs and more server side development?
- https://jsfiddle.net/: older but similar to Codepen. Similar features but maybe less actively maintained and less commercial/company support behind it.
- https://jsbin.com/: Similar to JS Fiddle but not actively maintained?
- Hello, Colaboratory
- Overview of Colaboratory Features
- Importing Libraries
- External data: Drive, Sheets, and Cloud Storage
- Getting Started with BigQuery
- Tensorflow with GPU
- Charts in Colaboratory
- Markdown Guide
- MusicVAE
Here are some fun interactive embedding visualizations from Google's AI Experiments, hopefully they also give some intuition about high dimensional spaces and what embedding does to them:
Here are some examples/tutorials on creating generative, procedural, or algorithmic art/designs using code.
- Making Generative Art with Simple Mathematics
- https://medium.com/@zachlieberman/daily-sketches-in-2017-1b4234b0615d
- Mike Bostock: Visualizing Algorithms (with Eyeo talk)
- Generative Examples in Processing
- Book of Shaders: Generative Designs
- Programming Design Systems
- Generative Music
- Procedural Content Generation in Games
A non-exhaustive list of people doing interesting things at the intersection of art, ML, and design.
- Memo Akten (website and blog)
- Rebecca Fiebrink
- Tom White
- Hannah Davis
- Mario Klingemann
- Robbie Barrat
- Mimi Onuoha
- Gene Kogan
- Allison Parrish
- Refik Anadol
- Tega Brain
- Kyle McDonald
- Mike Tyka
- Lauren McCarthy
- Dan Shiffman
- Molly Wright Steenson
- Ken Goldberg
- Eric Paulos
- Golan Levin
- STUDIO for Creative Inquiry
- ITP
- Gray Area Foundation for the Arts
- Berkeley Center for New Media
- Goldsmiths
- Google Artists and Machine Intelligence
- Google Creative Lab
- The Lab at the Google Cultural Institute
- Sony CSL: Tokyo and Paris
- The Brown Institute
- UCLA Design Media Arts
This work by Jonathan Dinu is licensed under CC BY 4.0
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms.
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation.
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.