This is an unofficial JAX implementation of "Two-Stream Convolutional Networks for Dynamic Texture Synthesis (CVPR'18)"
Please see the author's repo here and cite them:
@inproceedings{tesfaldet2018,
author = {Matthew Tesfaldet and Marcus A. Brubaker and Konstantinos G. Derpanis},
title = {Two-Stream Convolutional Networks for Dynamic Texture Synthesis},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2018}
}
We require these libraries:
pip install -U "jax[cuda]" equinox optax tqdm pillow
Thus far, we can NOT fully figure out and stick with the configurations in the official repo, but it works anyway 😄.
We re-write the appearance / motion stream network and the proposed two-stream loss in the paper, in JAX code. Networks are built on top of equinox.
Pre-trained weights are ported from here (VGG) and here (optical flow network)
python two_stream_dyntex_syn.py --exemplar_path data/fish
X | fish | flames | escalator |
---|---|---|---|
Input | |||
Output |
Thanks all efforts put on making all mentioned repositories public.
We appreciate bug reports. I will fix them when I make time around.