Skip to content

Dataviz/add slider to python api #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Sep 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
81404de
feat: create a image slider dataviz
andrewizbatista Aug 31, 2022
4eaab95
cleanup the js code
andrewizbatista Aug 31, 2022
fcfce32
fix typos
andrewizbatista Aug 31, 2022
a950691
Removing JSON parse
JoaoLages Aug 31, 2022
368ebc5
add GeneratedImages
JoaoLages Aug 31, 2022
d01ac2c
move code to show method instead
JoaoLages Aug 31, 2022
40f5cdd
bump version
JoaoLages Aug 31, 2022
f0da855
update notebook
JoaoLages Aug 31, 2022
1c67a06
update readme with slider information
JoaoLages Aug 31, 2022
8e835d2
remove bounding box from last image
JoaoLages Aug 31, 2022
3a68847
change image_slider.gif
JoaoLages Aug 31, 2022
2e114bc
add a loading animation and fix a few minor issues
andrewizbatista Sep 1, 2022
4e4962a
Merge branch 'dataviz/image-slider' of https://github.com/JoaoLages/d…
JoaoLages Sep 1, 2022
c785666
fix typo
JoaoLages Sep 1, 2022
c0036f6
try to display loading before image slider
JoaoLages Sep 1, 2022
0e99cab
make loading enabled at initialization
andrewizbatista Sep 1, 2022
eb6ebc9
Merge branch 'dataviz/image-slider' of https://github.com/JoaoLages/d…
JoaoLages Sep 1, 2022
be62a92
try display_id=42
JoaoLages Sep 1, 2022
ce94031
Merge branch 'dataviz/add-slider-to-python-api' of https://github.com…
JoaoLages Sep 1, 2022
46727d5
try to display both loading and slider at once
JoaoLages Sep 1, 2022
74b35d5
Merge branch 'dataviz/add-slider-to-python-api' of https://github.com…
JoaoLages Sep 1, 2022
5000125
try with update display
JoaoLages Sep 1, 2022
1cf1f5b
Merge branch 'dataviz/add-slider-to-python-api' of https://github.com…
JoaoLages Sep 1, 2022
3f31789
Update README.md
JoaoLages Sep 1, 2022
48b37b1
Merge branch 'dataviz/add-slider-to-python-api' of https://github.com…
JoaoLages Sep 1, 2022
6b28aed
update notebook
JoaoLages Sep 1, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,8 @@ output['sample']

![](assets/corgi_eiffel_tower.png)

You can also check the image that the diffusion process generated in the end of each step.

For example, to see the image from step 10:
```python
output['all_samples_during_generation'][10]
```
![](assets/corgi_eiffel_tower_step10.png)
You can also check all the images that the diffusion process generated at the end of each step.
![](assets/image_slider.gif)

To check how a token in the input `prompt` influenced the generation, you can check the token attribution scores:
```python
Expand Down Expand Up @@ -132,7 +127,7 @@ The token attributions are now computed only for the area specified in the image
Check other functionalities and more implementation examples in [here](https://github.com/JoaoLages/diffusers-interpret/blob/main/notebooks/).

## Future Development
- [ ] Add interactive display of all the images that were generated in the diffusion process
- [x] ~~Add interactive display of all the images that were generated in the diffusion process~~
- [ ] Add interactive bounding-box and token attributions visualization
- [ ] Add unit tests
- [ ] Add example for `diffusers_interpret.LDMTextToImagePipelineExplainer`
Expand All @@ -141,3 +136,7 @@ Check other functionalities and more implementation examples in [here](https://g

## Contributing
Feel free to open an [Issue](https://github.com/JoaoLages/diffusers-interpret/issues) or create a [Pull Request](https://github.com/JoaoLages/diffusers-interpret/pulls) and let's get started 🚀

## Credits

A special thanks to [@andrewizbatista](https://github.com/andrewizbatista) for creating a great [image slider](https://github.com/JoaoLages/diffusers-interpret/pull/1) to show all the generated images during diffusion! 💪
Binary file added assets/image_slider.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
245 changes: 147 additions & 98 deletions notebooks/stable-diffusion-example.ipynb

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

setup(
name='diffusers-interpret',
version='0.1.0',
version='0.2.0',
description='diffusers-interpret: model explainability for 🤗 Diffusers',
long_description=long_description,
long_description_content_type='text/markdown',
Expand Down
226 changes: 226 additions & 0 deletions src/diffusers_interpret/dataviz/image-slider/css/index.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
:root {
--animation-time: 100ms;
--image-size: 296px;
--loading-margin: 340px;
--error-margin: 390px;
--border-radius: 4px;
--color-primary: #ff6347;
--color-primary-hover: #e46b55;
--color-primary-active: #9acd32;
--color-primary-disabled: #aa8983;
--color-loading: #aa8983;
}

html {
font-family: system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
Oxygen, Ubuntu, Cantarell, "Open Sans", "Helvetica Neue", sans-serif;
}

#slider {
display: none;
flex-direction: row;
justify-content: flex-start;
align-items: flex-start;
user-select: none;
}

#error {
display: none;
flex-direction: row;
justify-content: flex-start;
align-items: center;
user-select: none;
height: 200px;
padding-left: var(--error-margin);
}

#error span {
font-size: 1.2rem;
color: var(--color-primary);
font-weight: bold;
letter-spacing: 0.5px;
}

#loading {
display: flex;
flex-direction: row;
justify-content: flex-start;
align-items: center;
user-select: none;
height: 200px;
opacity: 0.5;
padding-left: var(--loading-margin);
-webkit-animation: pulsate 1s ease-out;
-moz-animation: pulsate 1s ease-out;
-ms-animation: pulsate 1s ease-out;
-o-animation: pulsate 1s ease-out;
animation: pulsate 1s ease-out;
-webkit-animation-iteration-count: infinite;
-moz-animation-iteration-count: infinite;
-ms-animation-iteration-count: infinite;
-o-animation-iteration-count: infinite;
animation-iteration-count: infinite;
}

#loading span {
font-size: 1rem;
margin-left: 0.5rem;
color: var(--color-loading);
letter-spacing: 1px;
}

.slider-item {
padding: 5px;
text-align: center;
}

.slider-image {
display: block;
width: var(--image-size);
height: var(--image-size);
margin-bottom: 10px;
background-color: #222;
background-position: center;
background-repeat: no-repeat;
background-size: contain;
-webkit-transition: all var(--animation-time) linear;
-moz-transition: all var(--animation-time) linear;
-ms-transition: all var(--animation-time) linear;
-o-transition: all var(--animation-time) linear;
transition: all var(--animation-time) linear;
}

.slider-title {
display: block;
font-size: 1rem;
font-weight: bold;
user-select: none;
}

.slider-iteration {
display: block;
font-size: 0.8rem;
margin-top: 6px;
}

.slide-actions {
display: flex;
flex-direction: row;
justify-content: space-between;
align-items: center;
background-color: var(--color-primary);
border-radius: var(--border-radius);
color: #fff;
}

button {
color: #fff;
background-color: var(--color-primary);
border: 0;
font-size: 0.8rem;
padding-top: 14px;
padding-bottom: 14px;
cursor: pointer;
user-select: none;
-webkit-transition: all var(--animation-time) linear;
-moz-transition: all var(--animation-time) linear;
-ms-transition: all var(--animation-time) linear;
-o-transition: all var(--animation-time) linear;
transition: all var(--animation-time) linear;
}

button:disabled {
cursor: default;
background-color: var(--color-primary-disabled) !important;
}

button:hover {
background-color: var(--color-primary-hover);
}

button:active {
background-color: var(--color-primary-active);
}

.slider-iterations {
font-size: 0.8rem;
font-family: monospace;
user-select: none;
cursor: default;
}

#slider-action-prev {
border-top-left-radius: var(--border-radius);
border-bottom-left-radius: var(--border-radius);
padding-left: 12px;
padding-right: 18px;
}

#slider-action-next {
border-top-right-radius: var(--border-radius);
border-bottom-right-radius: var(--border-radius);
padding-left: 18px;
padding-right: 12px;
}

#loading-ripple {
display: inline-block;
position: relative;
width: 80px;
height: 80px;
}

#loading-ripple div {
position: absolute;
border: 4px solid var(--color-loading);
opacity: 1;
border-radius: 50%;
animation: ripple 1s cubic-bezier(0, 0.2, 0.8, 1) infinite;
}

#loading-ripple div:nth-child(2) {
animation-delay: -0.5s;
}

@keyframes ripple {
0% {
top: 36px;
left: 36px;
width: 0;
height: 0;
opacity: 0;
}
4.9% {
top: 36px;
left: 36px;
width: 0;
height: 0;
opacity: 0;
}
5% {
top: 36px;
left: 36px;
width: 0;
height: 0;
opacity: 1;
}
100% {
top: 0px;
left: 0px;
width: 72px;
height: 72px;
opacity: 0;
}
}

@keyframes pulsate {
0% {
opacity: 0.5;
}
50% {
opacity: 1;
}
100% {
opacity: 0.5;
}
}
43 changes: 43 additions & 0 deletions src/diffusers_interpret/dataviz/image-slider/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
<html>
<head>
<title>Image Slider</title>
<link href="css/index.css" rel="stylesheet" />
<meta charset="UTF-8" />
</head>
<body>
<section id="loading">
<div id="loading-ripple">
<div></div>
<div></div>
</div>
<span id="loading-name" class="pulsate">Preparing Data...</span>
</section>
<section id="error">
<span>Unexpected Error</span>
</section>
<section id="slider">
<div class="slider-item">
<div id="slider-image-first" class="slider-image"></div>
<span class="slider-title">First Iteration</span>
</div>
<div class="slider-item">
<div id="slider-image-current" class="slider-image"></div>
<div class="slide-actions">
<button id="slider-action-prev">◄ Prev</button>
<span class="slider-iterations">
<span id="slider-iterations-current">0</span>
/
<span id="slider-iterations-final">0</span>
</span>
<button id="slider-action-next">Next ►</button>
</div>
</div>
<div class="slider-item">
<div id="slider-image-final" class="slider-image"></div>
<span class="slider-title">Final Iteration</span>
</div>
</section>
<!-- INSERT STARTING SCRIPT HERE -->
<script type="text/javascript" src="js/index.js"></script>
</body>
</html>
Loading