diff --git a/00_core.ipynb b/00_core.ipynb index 18a160b..fa8d23e 100644 --- a/00_core.ipynb +++ b/00_core.ipynb @@ -93,7 +93,7 @@ "\n", "For now, calling the `queryFlow` process directly is the simplest gateway.\n", "\n", - "*Meaningless. Explain how to use it closer to where it is implemented. Also show an example of it in use. Terrible sentence really." + "*Meaningless. Explain how to use it closer to where it is implemented. Also show an example of it in use. Terrible sentence really.*" ] }, { diff --git a/README.md b/README.md index 4a7bcf5..2b5f8c6 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,15 @@ > Search over large image datasets with natural language and computer vision! +*^ that is so nerdy. can we do something like "Local-first AI image search"? or is that still too opaque* + ![meme about having too many memes](images/E2GoeMyWEAAkcLz.jpeg) -You know the problem. You have a huge folder of images: memes, screenshots, datasets, product photos, inspo albums, anything. You know that somewhere in that folder is the exact image you want, but you can't remember the filename or what day you saved it. There's nothing you can do. You have to scroll through the folder, skimming hundreds of thumbnails, hoping you don't accidentally miss it, hoping you'll recognize it when you do see it. +The problem: you have a huge folder of images. Memes, screenshots, datasets, product photos, inspo albums, anything. You know that somewhere in that folder is the exact image you want, but you can't remember the filename or what day you saved it. There's nothing you can do buit scroll through the folder, skimming hundreds of thumbnails, hoping you don't accidentally miss it, hoping you'll recognize it when you do see it. -Humans can do this amazingly well, all things considered. But even with computers, local image search is still pretty much a manual effort - you're still sorting through folders of images, like an archivist of old. +Humans do this amazingly well. But even with computers, local image search is still a manual effort - you're still sorting through folders of images, like an archivist of old. -Until now. +**Until memery**. The `memery` package provides natural language search over local images. You can use it to search for things like "a line drawing of a woman facing to the left" and get _reasonably good results!_ @@ -45,16 +47,43 @@ Outline: ## Install -The necessary CLIP and torch packages will be installed by pip. You might want to make sure you have a sane CUDA environment before and after this step if you're trying to use GPU. If you don't have a GPU, `memery` should still work on your CPU. +The necessary CLIP and torch packages will be installed by pip. + +*what does this mean? why are you telling me this? what if they aren't?* + +You might want to make sure you have a sane CUDA environment before and after this step if you're trying to use GPU. If you don't have a GPU, `memery` should still work on your CPU. + +*this is backwards and also impenetrable if you're not a nerd* + +If you have any trouble please **open an issue on Github**! I want to make this package useful for as many people as possible. Help me know what's going wrong :) + +*still true, pretty clear. use emoji or go home though* -If you have any trouble please **open an issue on Github**! I want to make this package useful for as many people as possible. Help me know what's going wrong :) +*This is the crucial command, huh? Seems a little far down the page. Maybe a quick-start section earlier, then a more in-depth Installation section here* `pip install memery` -If you don't have a GPU for PyTorch, this command might help - +If you don't have a GPU for PyTorch, this command might help: `pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html` +*did you scrape this from stackoverflow? or the CLIP source code or what. it looks very specific and arbitrary. I think it's meant to install pytorch CPU version, but the average user shouldn't have to know that! if you are trying to use CUDA and GPU it should be an easy toggle, and if you're a regular person wihtout a data science project you shouldn't have to go through a bunch of weird command line stuff.* + +*In fact, how can we package this in a way that you don't have to type a single command? There has to be ways to wrap up a whole python setup and install it on a different computer, with all the same dependencies. Right? Ugh, unfortunately this is Python, so it might not exist. But I can at least scale the difficulty level with the user story.* + +*Yes, user stories. We have to think about the interface between human and computer as another API, and look for the endpoints we can serve to different users with different protocols.* + +*This means thinking about the reasons people come to the program, and how they expect it to respond. Currently I see three user stories:* + * *i have images and want to search them with a GUI app* + * *streamlit GUI* + * *i have a program/workflow and want to use image search as one part of it* + * *CLI tool* + * *python module* + * *jupyter GUI* + * *i want to improve on and/or contribute to memery development* + * *clone the repo* + + +*Each one has an increasing level of complexity and expected knowledge.* ## How to use @@ -66,7 +95,10 @@ The browser GUI is a Streamlit app. You can run it from the command line with or set up a desktop shortcut to use it from your menu. -If you're in a Jupyter environment, you can summon the GUI directly into an output cell like this: + +*Why do I have to use a CLI to get a GUI? This is very annoying. In any case, this does not provide instructuions on how to use the GUI or any of its particular quirks. There should be a screenshot, also* + +If you're in a Jupyter environment, you can summon an ipywidgets GUI directly into an output cell like this: ```python @@ -80,13 +112,19 @@ display(app) ``` - + + +*This doesn't transfer to the README, so it should be fixed or be scrapped. Screenshots would work fine. Also I'm not sure if the interactive widgets would make sense without a kernel behind them anyway.* ### Use CLI From the command line, you can use `memery` on any folder and it will search for images recursively, returning a list object to stdout. +*Why is it that behavior? Is it possible to make it not recursive? What is the usual behavior of a search in POSIX? What is the use case for a CLI memery: is it shell scripts? in-console image browsing? risk-tolerant batch modification? Think about this further* + +*at least you can control how much output it spews to your terminal* + Pass the --n flag to control how many images are returned (default 10). `memery recall PATH/TO/IMAGE/FOLDER 'query' --n 20 @@ -94,8 +132,14 @@ Pass the --n flag to control how many images are returned (default 10). ### Use as a library +*the following paragrah cannot be comprehended by any human mind. if you can even read to the end of it you should go touch grass* + Simply use `queryFlow` to search over a folder recursively! The folder will be indexed, if an index doesn't already exist. Then any new images will be CLIP-encoded, an Annoy treemap built, and a list of ranked filenames returned. +*okay, we survived. What are we supposed to say here instead?* + +*I think what he's trying to get at is explaining the underlying mechanism of the search. Really all we need here is a function signature, as anyone who's going to use it as a library will probably also be looking at the docs for further information. Have a link here too, that goes to the `core` file where the main flows and executors are explained.* + ```python from memery.core import queryFlow from memery.gui import get_grid @@ -109,6 +153,10 @@ print(ranked[:5]) --- +*Okay, now we need a whole part about development. + +--- + *Compile this notebook* ```python diff --git a/docs/_data/sidebars/home_sidebar.yml b/docs/_data/sidebars/home_sidebar.yml index 4f1fd60..d0375fe 100644 --- a/docs/_data/sidebars/home_sidebar.yml +++ b/docs/_data/sidebars/home_sidebar.yml @@ -39,6 +39,9 @@ entries: - output: web,pdf title: Streamlit app url: streamlit_app.html + - output: web,pdf + title: memery + url: index-Copy1.html output: web title: memery output: web diff --git a/docs/core.html b/docs/core.html index 672345c..cb30b09 100644 --- a/docs/core.html +++ b/docs/core.html @@ -32,6 +32,10 @@

This is the core module, which loads the CLIP model and the encodings of all the images in the folder, then tokenizes the search text or image, and finally returns a sorted list of filenames.

+

Express this as a process and nouns, instead of a blob sentence. The next part doesn't count because it's a whole new heading and it's crufted with nerd talk. Say clearly what the core flow is here. Something like:

+

Memery takes a folder of images, and a search query, and returns a list of nearest neighbors to the query.

+

The query flow and index flow can be used separately, but by default the query flow calls the indexing flow on each search. Image encodings are saved to disk. Only new images will be encoded with each indexing.

+

d

@@ -62,6 +66,7 @@

Modular flow systemModular flow system

Usage

For now, calling the queryFlow process directly is the simplest gateway.

+

Meaningless. Explain how to use it closer to where it is implemented. Also show an example of it in use. Terrible sentence really.

@@ -89,6 +95,7 @@

Flows

The indexFlow checks the local directory for files with image extensions, loads the archive, splits out any new images to be encoded and encodes them, then finally builds a treemap and saves it along with the new archive. It returns a tuple with the locations of the archive and treemap.

+

Split this up into small chunks with code tests. No blob sentences!

@@ -181,6 +188,7 @@

indexFlow

The queryFlow takes a path and a query, checks for an index, loads it and searches through the treemap if it exists, and calls indexFlow to index it first if it hasn't been.

+

BLOB!

@@ -216,6 +224,18 @@

queryFlow
+
+

Then what?! What are the limitations of this system? What are its options? What configuration can i do if i'm a power user? Why did you organize things this way instead of a different way?

+

This, and probably each of the following notebooks, would benefit from a small recording session where I try to explain it to an imaginary audience. So that I can get the narrative of how it works, and then arrange the code around that.

+ +
+
+

+ + diff --git a/docs/index-Copy1.html b/docs/index-Copy1.html new file mode 100644 index 0000000..c02810e --- /dev/null +++ b/docs/index-Copy1.html @@ -0,0 +1,329 @@ +--- + +title: memery + + +keywords: fastai +sidebar: home_sidebar + +summary: "Search over large image datasets with natural language and computer vision!" +description: "Search over large image datasets with natural language and computer vision!" +nb_path: "index-Copy1.ipynb" +--- + + +
+ + {% raw %} + +
+ +
+ {% endraw %} + +
+
+

^ that is so nerdy. can we do something like "Local-first AI image search"? or is that still too opaque

+

meme about having too many memes

+

The problem: you have a huge folder of images. Memes, screenshots, datasets, product photos, inspo albums, anything. You know that somewhere in that folder is the exact image you want, but you can't remember the filename or what day you saved it. There's nothing you can do buit scroll through the folder, skimming hundreds of thumbnails, hoping you don't accidentally miss it, hoping you'll recognize it when you do see it.

+

Humans do this amazingly well. But even with computers, local image search is still a manual effort - you're still sorting through folders of images, like an archivist of old.

+

Until memery.

+

The memery package provides natural language search over local images. You can use it to search for things like "a line drawing of a woman facing to the left" and get reasonably good results!

+

You can do this over thousands of images (it's not optimized for performance yet, but search times scale well under O(n)).

+

You can view the images in a browser GUI, or pipe them through command line tools.

+

You can use memery or its modules in Jupyter notebooks, including GUI functions!

+

Under the hood, memery makes use of CLIP, the Contrastive Language-Image Pretraining transformer, released by OpenAI in 2021. CLIP trains a vision transformer and a language transformer to find the same latent space for images and their captions. This makes it perfect for the purpose of natural language image search. CLIP is a giant achievement, and memery stands on its shoulders.

+ +
+
+
+
+
+

Outline:

+
    +
  • Usage
      +
    • Install locally
      • +
    • Use GUI
      • +
    • Use CLI
      • +
    • Use in Jupyter
      • +
    • Use the library
      • +
    +
    • +
  • Development
      +
    • Notebook-driven development
      • +
    • Pull the repo
      • +
    • Branch and install
      • +
    • Notebook-driven development
      • +
    • Change the notebooks
      • +
    • Test the notebooks
      • +
    • Notebook-driven development
      • +
    • Tangle the source code
      • +
    • Weave the documentation
      • +
    +
    • +
  • Contributing
      +
    • Who works on this project
      • +
    • How you can help
      • +
    • What we don't do
      • +
    • Thanks
      • +
    +
    • +
+ +
+
+
+
+
+

Install

The necessary CLIP and torch packages will be installed by pip.

+

what does this mean? why are you telling me this? what if they aren't?

+

You might want to make sure you have a sane CUDA environment before and after this step if you're trying to use GPU. If you don't have a GPU, memery should still work on your CPU.

+

this is backwards and also impenetrable if you're not a nerd

+

If you have any trouble please open an issue on Github! I want to make this package useful for as many people as possible. Help me know what's going wrong :)

+

still true, pretty clear. use emoji or go home though

+

This is the crucial command, huh? Seems a little far down the page. Maybe a quick-start section earlier, then a more in-depth Installation section here

+

pip install memery

+

If you don't have a GPU for PyTorch, this command might help: +pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

+

did you scrape this from stackoverflow? or the CLIP source code or what. it looks very specific and arbitrary. I think it's meant to install pytorch CPU version, but the average user shouldn't have to know that! if you are trying to use CUDA and GPU it should be an easy toggle, and if you're a regular person wihtout a data science project you shouldn't have to go through a bunch of weird command line stuff.

+

In fact, how can we package this in a way that you don't have to type a single command? There has to be ways to wrap up a whole python setup and install it on a different computer, with all the same dependencies. Right? Ugh, unfortunately this is Python, so it might not exist. But I can at least scale the difficulty level with the user story.

+ +
+
+
+
+
+

Yes, user stories. We have to think about the interface between human and computer as another API, and look for the endpoints we can serve to different users with different protocols.

+

This means thinking about the reasons people come to the program, and how they expect it to respond. Currently I see three user stories:

+
    +
  • i have images and want to search them with a GUI app
      +
    • streamlit GUI
      • +
    +
    • +
  • i have a program/workflow and want to use image search as one part of it
      +
    • CLI tool
      • +
    • python module
      • +
    • jupyter GUI
      • +
    +
    • +
  • i want to improve on and/or contribute to memery development
      +
    • clone the repo
      • +
    +
    • +
+

Each one has an increasing level of complexity and expected knowledge.

+ +
+
+
+
+
+

How to use

+
+
+
+
+
+

Use GUI

The browser GUI is a Streamlit app. You can run it from the command line with

+

memery serve

+

or set up a desktop shortcut to use it from your menu.

+

Why do I have to use a CLI to get a GUI? This is very annoying. In any case, this does not provide instructuions on how to use the GUI or any of its particular quirks. There should be a screenshot, also

+

If you're in a Jupyter environment, you can summon an ipywidgets GUI directly into an output cell like this:

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
from memery.gui import appPage
+
+ +
+
+
+ +
+ {% endraw %} + + {% raw %} + +
+
+ +
+
+
app = appPage()
+display(app)
+
+ +
+
+
+ +
+
+ +
+ + + +
+
<memery.gui.appPage at 0x7ff3072db110>
+
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

This doesn't transfer to the README, so it should be fixed or be scrapped. Screenshots would work fine. Also I'm not sure if the interactive widgets would make sense without a kernel behind them anyway.

+ +
+
+
+
+
+

Use CLI

From the command line, you can use memery on any folder and it will search for images recursively, returning a list object to stdout.

+

Why is it that behavior? Is it possible to make it not recursive? What is the usual behavior of a search in POSIX? What is the use case for a CLI memery: is it shell scripts? in-console image browsing? risk-tolerant batch modification? Think about this further

+

at least you can control how much output it spews to your terminal

+

Pass the --n flag to control how many images are returned (default 10).

+

memery recall PATH/TO/IMAGE/FOLDER 'query' --n 20

+ +
+
+
+
+
+

Use as a library

+
+
+
+
+
+

the following paragrah cannot be comprehended by any human mind. if you can even read to the end of it you should go touch grass

+

Simply use queryFlow to search over a folder recursively! The folder will be indexed, if an index doesn't already exist. Then any new images will be CLIP-encoded, an Annoy treemap built, and a list of ranked filenames returned.

+

okay, we survived. What are we supposed to say here instead?

+

I think what he's trying to get at is explaining the underlying mechanism of the search. Really all we need here is a function signature, as anyone who's going to use it as a library will probably also be looking at the docs for further information. Have a link here too, that goes to the core file where the main flows and executors are explained.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
from memery.core import queryFlow
+from memery.gui import get_grid
+
+ +
+
+
+ +
+ {% endraw %} + + {% raw %} + +
+
+ +
+
+
ranked = queryFlow('./images', 'dad joke')
+
+print(ranked[:5])
+
+ +
+
+
+ +
+ {% endraw %} + +
+
+
+

*Okay, now we need a whole part about development.

+ +
+
+
+
+
+
+

Compile this notebook

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
from nbdev.export import notebook2script; notebook2script()
+
+ +
+
+
+ +
+
+ +
+ +
+
Converted 00_core.ipynb.
+Converted 01_loader.ipynb.
+Converted 02_crafter.ipynb.
+Converted 03_encoder.ipynb.
+Converted 04_indexer.ipynb.
+Converted 05_ranker.ipynb.
+Converted 06_fileutils.ipynb.
+Converted 07_cli.ipynb.
+Converted 08_jupyter_gui.ipynb.
+Converted 09_streamlit_app.ipynb.
+Converted index.ipynb.
+
+
+
+ +
+
+ +
+ {% endraw %} + +
+ + + + diff --git a/docs/index.html b/docs/index.html index 8e22e3d..5b94996 100644 --- a/docs/index.html +++ b/docs/index.html @@ -31,10 +31,11 @@
+

^ that is so nerdy. can we do something like "Local-first AI image search"? or is that still too opaque

meme about having too many memes

-

You know the problem. You have a huge folder of images: memes, screenshots, datasets, product photos, inspo albums, anything. You know that somewhere in that folder is the exact image you want, but you can't remember the filename or what day you saved it. There's nothing you can do. You have to scroll through the folder, skimming hundreds of thumbnails, hoping you don't accidentally miss it, hoping you'll recognize it when you do see it.

-

Humans can do this amazingly well, all things considered. But even with computers, local image search is still pretty much a manual effort - you're still sorting through folders of images, like an archivist of old.

-

Until now.

+

The problem: you have a huge folder of images. Memes, screenshots, datasets, product photos, inspo albums, anything. You know that somewhere in that folder is the exact image you want, but you can't remember the filename or what day you saved it. There's nothing you can do buit scroll through the folder, skimming hundreds of thumbnails, hoping you don't accidentally miss it, hoping you'll recognize it when you do see it.

+

Humans do this amazingly well. But even with computers, local image search is still a manual effort - you're still sorting through folders of images, like an archivist of old.

+

Until memery.

The memery package provides natural language search over local images. You can use it to search for things like "a line drawing of a woman facing to the left" and get reasonably good results!

You can do this over thousands of images (it's not optimized for performance yet, but search times scale well under O(n)).

You can view the images in a browser GUI, or pipe them through command line tools.

@@ -82,11 +83,43 @@
-

Install

The necessary CLIP and torch packages will be installed by pip. You might want to make sure you have a sane CUDA environment before and after this step if you're trying to use GPU. If you don't have a GPU, memery should still work on your CPU.

+

Install

The necessary CLIP and torch packages will be installed by pip.

+

what does this mean? why are you telling me this? what if they aren't?

+

You might want to make sure you have a sane CUDA environment before and after this step if you're trying to use GPU. If you don't have a GPU, memery should still work on your CPU.

+

this is backwards and also impenetrable if you're not a nerd

If you have any trouble please open an issue on Github! I want to make this package useful for as many people as possible. Help me know what's going wrong :)

+

still true, pretty clear. use emoji or go home though

+

This is the crucial command, huh? Seems a little far down the page. Maybe a quick-start section earlier, then a more in-depth Installation section here

pip install memery

-

If you don't have a GPU for PyTorch, this command might help

-

pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

+

If you don't have a GPU for PyTorch, this command might help: +pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

+

did you scrape this from stackoverflow? or the CLIP source code or what. it looks very specific and arbitrary. I think it's meant to install pytorch CPU version, but the average user shouldn't have to know that! if you are trying to use CUDA and GPU it should be an easy toggle, and if you're a regular person wihtout a data science project you shouldn't have to go through a bunch of weird command line stuff.

+

In fact, how can we package this in a way that you don't have to type a single command? There has to be ways to wrap up a whole python setup and install it on a different computer, with all the same dependencies. Right? Ugh, unfortunately this is Python, so it might not exist. But I can at least scale the difficulty level with the user story.

+ +
+
+
+
+
+

Yes, user stories. We have to think about the interface between human and computer as another API, and look for the endpoints we can serve to different users with different protocols.

+

This means thinking about the reasons people come to the program, and how they expect it to respond. Currently I see three user stories:

+
    +
  • i have images and want to search them with a GUI app
      +
    • streamlit GUI
      • +
    +
    • +
  • i have a program/workflow and want to use image search as one part of it
      +
    • CLI tool
      • +
    • python module
      • +
    • jupyter GUI
      • +
    +
    • +
  • i want to improve on and/or contribute to memery development
      +
    • clone the repo
      • +
    +
    • +
+

Each one has an increasing level of complexity and expected knowledge.

@@ -102,7 +135,8 @@

How to use Use GUI

The browser GUI is a Streamlit app. You can run it from the command line with

memery serve

or set up a desktop shortcut to use it from your menu.

-

If you're in a Jupyter environment, you can summon the GUI directly into an output cell like this:

+

Why do I have to use a CLI to get a GUI? This is very annoying. In any case, this does not provide instructuions on how to use the GUI or any of its particular quirks. There should be a screenshot, also

+

If you're in a Jupyter environment, you can summon an ipywidgets GUI directly into an output cell like this:

@@ -147,7 +181,7 @@

Use GUI

The

-
<memery.gui.appPage at 0x7f6c0e7c80d0>
+
<memery.gui.appPage at 0x7ff3072db110>
@@ -158,9 +192,18 @@

Use GUI

The {% endraw %} +

+
+

This doesn't transfer to the README, so it should be fixed or be scrapped. Screenshots would work fine. Also I'm not sure if the interactive widgets would make sense without a kernel behind them anyway.

+ +
+
+

Use CLI

From the command line, you can use memery on any folder and it will search for images recursively, returning a list object to stdout.

+

Why is it that behavior? Is it possible to make it not recursive? What is the usual behavior of a search in POSIX? What is the use case for a CLI memery: is it shell scripts? in-console image browsing? risk-tolerant batch modification? Think about this further

+

at least you can control how much output it spews to your terminal

Pass the --n flag to control how many images are returned (default 10).

memery recall PATH/TO/IMAGE/FOLDER 'query' --n 20

@@ -175,7 +218,10 @@

Use as a library
+

the following paragrah cannot be comprehended by any human mind. if you can even read to the end of it you should go touch grass

Simply use queryFlow to search over a folder recursively! The folder will be indexed, if an index doesn't already exist. Then any new images will be CLIP-encoded, an Annoy treemap built, and a list of ranked filenames returned.

+

okay, we survived. What are we supposed to say here instead?

+

I think what he's trying to get at is explaining the underlying mechanism of the search. Really all we need here is a function signature, as anyone who's going to use it as a library will probably also be looking at the docs for further information. Have a link here too, that goes to the core file where the main flows and executors are explained.

@@ -217,6 +263,14 @@

Use as a library
+
+
+

*Okay, now we need a whole part about development.

+ +
+
+

diff --git a/docs/sidebar.json b/docs/sidebar.json index a9ab23d..f7d95cd 100644 --- a/docs/sidebar.json +++ b/docs/sidebar.json @@ -10,6 +10,7 @@ "File Utils": "fileutils.html", "CLI": "cli.html", "GUI": "jupyter_gui.html", - "Streamlit app": "streamlit_app.html" + "Streamlit app": "streamlit_app.html", + "memery": "index-Copy1.html" } } \ No newline at end of file diff --git a/images/jupyter-screenshot.png b/images/jupyter-screenshot.png new file mode 100644 index 0000000..3450579 Binary files /dev/null and b/images/jupyter-screenshot.png differ diff --git a/images/memery.ann b/images/memery.ann index ab99f07..ebfc7af 100644 Binary files a/images/memery.ann and b/images/memery.ann differ diff --git a/images/memery.pt b/images/memery.pt index b599bc1..28186ab 100644 Binary files a/images/memery.pt and b/images/memery.pt differ diff --git a/images/Envato-Elements.png b/images/memes/Envato-Elements.png similarity index 100% rename from images/Envato-Elements.png rename to images/memes/Envato-Elements.png diff --git a/images/Father-and-son-having-fun-at-the-breakfast-table.jpg b/images/memes/Father-and-son-having-fun-at-the-breakfast-table.jpg similarity index 100% rename from images/Father-and-son-having-fun-at-the-breakfast-table.jpg rename to images/memes/Father-and-son-having-fun-at-the-breakfast-table.jpg diff --git a/images/Shrek_screenshot.jpg b/images/memes/Shrek_screenshot.jpg similarity index 100% rename from images/Shrek_screenshot.jpg rename to images/memes/Shrek_screenshot.jpg diff --git a/images/Wholesome-Meme-1.jpg b/images/memes/Wholesome-Meme-1.jpg similarity index 100% rename from images/Wholesome-Meme-1.jpg rename to images/memes/Wholesome-Meme-1.jpg diff --git a/images/Wholesome-Meme-10.jpg b/images/memes/Wholesome-Meme-10.jpg similarity index 100% rename from images/Wholesome-Meme-10.jpg rename to images/memes/Wholesome-Meme-10.jpg diff --git a/images/Wholesome-Meme-12.jpg b/images/memes/Wholesome-Meme-12.jpg similarity index 100% rename from images/Wholesome-Meme-12.jpg rename to images/memes/Wholesome-Meme-12.jpg diff --git a/images/Wholesome-Meme-13.jpg b/images/memes/Wholesome-Meme-13.jpg similarity index 100% rename from images/Wholesome-Meme-13.jpg rename to images/memes/Wholesome-Meme-13.jpg diff --git a/images/Wholesome-Meme-14.jpg b/images/memes/Wholesome-Meme-14.jpg similarity index 100% rename from images/Wholesome-Meme-14.jpg rename to images/memes/Wholesome-Meme-14.jpg diff --git a/images/Wholesome-Meme-15.jpg b/images/memes/Wholesome-Meme-15.jpg similarity index 100% rename from images/Wholesome-Meme-15.jpg rename to images/memes/Wholesome-Meme-15.jpg diff --git a/images/Wholesome-Meme-16.jpg b/images/memes/Wholesome-Meme-16.jpg similarity index 100% rename from images/Wholesome-Meme-16.jpg rename to images/memes/Wholesome-Meme-16.jpg diff --git a/images/Wholesome-Meme-17.jpg b/images/memes/Wholesome-Meme-17.jpg similarity index 100% rename from images/Wholesome-Meme-17.jpg rename to images/memes/Wholesome-Meme-17.jpg diff --git a/images/Wholesome-Meme-18.jpg b/images/memes/Wholesome-Meme-18.jpg similarity index 100% rename from images/Wholesome-Meme-18.jpg rename to images/memes/Wholesome-Meme-18.jpg diff --git a/images/Wholesome-Meme-21.jpg b/images/memes/Wholesome-Meme-21.jpg similarity index 100% rename from images/Wholesome-Meme-21.jpg rename to images/memes/Wholesome-Meme-21.jpg diff --git a/images/Wholesome-Meme-22.jpg b/images/memes/Wholesome-Meme-22.jpg similarity index 100% rename from images/Wholesome-Meme-22.jpg rename to images/memes/Wholesome-Meme-22.jpg diff --git a/images/Wholesome-Meme-23.jpg b/images/memes/Wholesome-Meme-23.jpg similarity index 100% rename from images/Wholesome-Meme-23.jpg rename to images/memes/Wholesome-Meme-23.jpg diff --git a/images/Wholesome-Meme-25.jpg b/images/memes/Wholesome-Meme-25.jpg similarity index 100% rename from images/Wholesome-Meme-25.jpg rename to images/memes/Wholesome-Meme-25.jpg diff --git a/images/Wholesome-Meme-27.jpg b/images/memes/Wholesome-Meme-27.jpg similarity index 100% rename from images/Wholesome-Meme-27.jpg rename to images/memes/Wholesome-Meme-27.jpg diff --git a/images/Wholesome-Meme-28.jpg b/images/memes/Wholesome-Meme-28.jpg similarity index 100% rename from images/Wholesome-Meme-28.jpg rename to images/memes/Wholesome-Meme-28.jpg diff --git a/images/Wholesome-Meme-29.jpg b/images/memes/Wholesome-Meme-29.jpg similarity index 100% rename from images/Wholesome-Meme-29.jpg rename to images/memes/Wholesome-Meme-29.jpg diff --git a/images/Wholesome-Meme-3.jpg b/images/memes/Wholesome-Meme-3.jpg similarity index 100% rename from images/Wholesome-Meme-3.jpg rename to images/memes/Wholesome-Meme-3.jpg diff --git a/images/Wholesome-Meme-31.jpg b/images/memes/Wholesome-Meme-31.jpg similarity index 100% rename from images/Wholesome-Meme-31.jpg rename to images/memes/Wholesome-Meme-31.jpg diff --git a/images/Wholesome-Meme-33.jpg b/images/memes/Wholesome-Meme-33.jpg similarity index 100% rename from images/Wholesome-Meme-33.jpg rename to images/memes/Wholesome-Meme-33.jpg diff --git a/images/Wholesome-Meme-34.jpg b/images/memes/Wholesome-Meme-34.jpg similarity index 100% rename from images/Wholesome-Meme-34.jpg rename to images/memes/Wholesome-Meme-34.jpg diff --git a/images/Wholesome-Meme-35.jpg b/images/memes/Wholesome-Meme-35.jpg similarity index 100% rename from images/Wholesome-Meme-35.jpg rename to images/memes/Wholesome-Meme-35.jpg diff --git a/images/Wholesome-Meme-36.jpg b/images/memes/Wholesome-Meme-36.jpg similarity index 100% rename from images/Wholesome-Meme-36.jpg rename to images/memes/Wholesome-Meme-36.jpg diff --git a/images/Wholesome-Meme-39.jpg b/images/memes/Wholesome-Meme-39.jpg similarity index 100% rename from images/Wholesome-Meme-39.jpg rename to images/memes/Wholesome-Meme-39.jpg diff --git a/images/Wholesome-Meme-4.jpg b/images/memes/Wholesome-Meme-4.jpg similarity index 100% rename from images/Wholesome-Meme-4.jpg rename to images/memes/Wholesome-Meme-4.jpg diff --git a/images/Wholesome-Meme-40.jpg b/images/memes/Wholesome-Meme-40.jpg similarity index 100% rename from images/Wholesome-Meme-40.jpg rename to images/memes/Wholesome-Meme-40.jpg diff --git a/images/Wholesome-Meme-40.png b/images/memes/Wholesome-Meme-40.png similarity index 100% rename from images/Wholesome-Meme-40.png rename to images/memes/Wholesome-Meme-40.png diff --git a/images/Wholesome-Meme-41.jpg b/images/memes/Wholesome-Meme-41.jpg similarity index 100% rename from images/Wholesome-Meme-41.jpg rename to images/memes/Wholesome-Meme-41.jpg diff --git a/images/Wholesome-Meme-42.jpg b/images/memes/Wholesome-Meme-42.jpg similarity index 100% rename from images/Wholesome-Meme-42.jpg rename to images/memes/Wholesome-Meme-42.jpg diff --git a/images/Wholesome-Meme-44.png b/images/memes/Wholesome-Meme-44.png similarity index 100% rename from images/Wholesome-Meme-44.png rename to images/memes/Wholesome-Meme-44.png diff --git a/images/Wholesome-Meme-45.jpg b/images/memes/Wholesome-Meme-45.jpg similarity index 100% rename from images/Wholesome-Meme-45.jpg rename to images/memes/Wholesome-Meme-45.jpg diff --git a/images/Wholesome-Meme-5.jpg b/images/memes/Wholesome-Meme-5.jpg similarity index 100% rename from images/Wholesome-Meme-5.jpg rename to images/memes/Wholesome-Meme-5.jpg diff --git a/images/Wholesome-Meme-57.jpg b/images/memes/Wholesome-Meme-57.jpg similarity index 100% rename from images/Wholesome-Meme-57.jpg rename to images/memes/Wholesome-Meme-57.jpg diff --git a/images/Wholesome-Meme-59.jpg b/images/memes/Wholesome-Meme-59.jpg similarity index 100% rename from images/Wholesome-Meme-59.jpg rename to images/memes/Wholesome-Meme-59.jpg diff --git a/images/Wholesome-Meme-6.jpg b/images/memes/Wholesome-Meme-6.jpg similarity index 100% rename from images/Wholesome-Meme-6.jpg rename to images/memes/Wholesome-Meme-6.jpg diff --git a/images/Wholesome-Meme-60.jpg b/images/memes/Wholesome-Meme-60.jpg similarity index 100% rename from images/Wholesome-Meme-60.jpg rename to images/memes/Wholesome-Meme-60.jpg diff --git a/images/Wholesome-Meme-61.png b/images/memes/Wholesome-Meme-61.png similarity index 100% rename from images/Wholesome-Meme-61.png rename to images/memes/Wholesome-Meme-61.png diff --git a/images/Wholesome-Meme-63.jpg b/images/memes/Wholesome-Meme-63.jpg similarity index 100% rename from images/Wholesome-Meme-63.jpg rename to images/memes/Wholesome-Meme-63.jpg diff --git a/images/Wholesome-Meme-64.jpg b/images/memes/Wholesome-Meme-64.jpg similarity index 100% rename from images/Wholesome-Meme-64.jpg rename to images/memes/Wholesome-Meme-64.jpg diff --git a/images/Wholesome-Meme-65.jpg b/images/memes/Wholesome-Meme-65.jpg similarity index 100% rename from images/Wholesome-Meme-65.jpg rename to images/memes/Wholesome-Meme-65.jpg diff --git a/images/Wholesome-Meme-67.png b/images/memes/Wholesome-Meme-67.png similarity index 100% rename from images/Wholesome-Meme-67.png rename to images/memes/Wholesome-Meme-67.png diff --git a/images/Wholesome-Meme-68.jpg b/images/memes/Wholesome-Meme-68.jpg similarity index 100% rename from images/Wholesome-Meme-68.jpg rename to images/memes/Wholesome-Meme-68.jpg diff --git a/images/Wholesome-Meme-69.jpg b/images/memes/Wholesome-Meme-69.jpg similarity index 100% rename from images/Wholesome-Meme-69.jpg rename to images/memes/Wholesome-Meme-69.jpg diff --git a/images/Wholesome-Meme-7.jpg b/images/memes/Wholesome-Meme-7.jpg similarity index 100% rename from images/Wholesome-Meme-7.jpg rename to images/memes/Wholesome-Meme-7.jpg diff --git a/images/Wholesome-Meme-70.jpg b/images/memes/Wholesome-Meme-70.jpg similarity index 100% rename from images/Wholesome-Meme-70.jpg rename to images/memes/Wholesome-Meme-70.jpg diff --git a/images/Wholesome-Meme-71.jpg b/images/memes/Wholesome-Meme-71.jpg similarity index 100% rename from images/Wholesome-Meme-71.jpg rename to images/memes/Wholesome-Meme-71.jpg diff --git a/images/Wholesome-Meme-72.jpg b/images/memes/Wholesome-Meme-72.jpg similarity index 100% rename from images/Wholesome-Meme-72.jpg rename to images/memes/Wholesome-Meme-72.jpg diff --git a/images/Wholesome-Meme-73.png b/images/memes/Wholesome-Meme-73.png similarity index 100% rename from images/Wholesome-Meme-73.png rename to images/memes/Wholesome-Meme-73.png diff --git a/images/Wholesome-Meme-74.jpg b/images/memes/Wholesome-Meme-74.jpg similarity index 100% rename from images/Wholesome-Meme-74.jpg rename to images/memes/Wholesome-Meme-74.jpg diff --git a/images/Wholesome-Meme-76.jpg b/images/memes/Wholesome-Meme-76.jpg similarity index 100% rename from images/Wholesome-Meme-76.jpg rename to images/memes/Wholesome-Meme-76.jpg diff --git a/images/Wholesome-Meme-77.jpg b/images/memes/Wholesome-Meme-77.jpg similarity index 100% rename from images/Wholesome-Meme-77.jpg rename to images/memes/Wholesome-Meme-77.jpg diff --git a/images/Wholesome-Meme-78.jpg b/images/memes/Wholesome-Meme-78.jpg similarity index 100% rename from images/Wholesome-Meme-78.jpg rename to images/memes/Wholesome-Meme-78.jpg diff --git a/images/Wholesome-Meme-8.jpg b/images/memes/Wholesome-Meme-8.jpg similarity index 100% rename from images/Wholesome-Meme-8.jpg rename to images/memes/Wholesome-Meme-8.jpg diff --git a/images/Wholesome-Meme-80.jpg b/images/memes/Wholesome-Meme-80.jpg similarity index 100% rename from images/Wholesome-Meme-80.jpg rename to images/memes/Wholesome-Meme-80.jpg diff --git a/images/Wholesome-Meme-81.jpg b/images/memes/Wholesome-Meme-81.jpg similarity index 100% rename from images/Wholesome-Meme-81.jpg rename to images/memes/Wholesome-Meme-81.jpg diff --git a/images/Wholesome-Meme-82.jpg b/images/memes/Wholesome-Meme-82.jpg similarity index 100% rename from images/Wholesome-Meme-82.jpg rename to images/memes/Wholesome-Meme-82.jpg diff --git a/images/Wholesome-Meme-84.jpg b/images/memes/Wholesome-Meme-84.jpg similarity index 100% rename from images/Wholesome-Meme-84.jpg rename to images/memes/Wholesome-Meme-84.jpg diff --git a/images/Wholesome-Meme-85.jpg b/images/memes/Wholesome-Meme-85.jpg similarity index 100% rename from images/Wholesome-Meme-85.jpg rename to images/memes/Wholesome-Meme-85.jpg diff --git a/images/Wholesome-Meme-86.jpg b/images/memes/Wholesome-Meme-86.jpg similarity index 100% rename from images/Wholesome-Meme-86.jpg rename to images/memes/Wholesome-Meme-86.jpg diff --git a/images/Wholesome-Meme-88.jpg b/images/memes/Wholesome-Meme-88.jpg similarity index 100% rename from images/Wholesome-Meme-88.jpg rename to images/memes/Wholesome-Meme-88.jpg diff --git a/images/Wholesome-Meme-89.jpg b/images/memes/Wholesome-Meme-89.jpg similarity index 100% rename from images/Wholesome-Meme-89.jpg rename to images/memes/Wholesome-Meme-89.jpg diff --git a/images/Wholesome-Meme-9.jpg b/images/memes/Wholesome-Meme-9.jpg similarity index 100% rename from images/Wholesome-Meme-9.jpg rename to images/memes/Wholesome-Meme-9.jpg diff --git a/images/Wholesome-Meme-97.jpg b/images/memes/Wholesome-Meme-97.jpg similarity index 100% rename from images/Wholesome-Meme-97.jpg rename to images/memes/Wholesome-Meme-97.jpg diff --git a/images/Wholesome-Meme-98.jpg b/images/memes/Wholesome-Meme-98.jpg similarity index 100% rename from images/Wholesome-Meme-98.jpg rename to images/memes/Wholesome-Meme-98.jpg diff --git a/images/Wholesome-Meme-99.jpg b/images/memes/Wholesome-Meme-99.jpg similarity index 100% rename from images/Wholesome-Meme-99.jpg rename to images/memes/Wholesome-Meme-99.jpg diff --git a/images/Wholesome-Meme.jpg b/images/memes/Wholesome-Meme.jpg similarity index 100% rename from images/Wholesome-Meme.jpg rename to images/memes/Wholesome-Meme.jpg diff --git a/images/braydon-anderson-wOHH-NUTvVc-unsplash-min.jpg b/images/memes/braydon-anderson-wOHH-NUTvVc-unsplash-min.jpg similarity index 100% rename from images/braydon-anderson-wOHH-NUTvVc-unsplash-min.jpg rename to images/memes/braydon-anderson-wOHH-NUTvVc-unsplash-min.jpg diff --git a/images/corrupted-file.jpeg b/images/memes/corrupted-file.jpeg similarity index 100% rename from images/corrupted-file.jpeg rename to images/memes/corrupted-file.jpeg diff --git a/images/cute-baby-touching-his-moms-face.jpeg b/images/memes/cute-baby-touching-his-moms-face.jpeg similarity index 100% rename from images/cute-baby-touching-his-moms-face.jpeg rename to images/memes/cute-baby-touching-his-moms-face.jpeg diff --git a/images/cute-dog-with-cupcake-P9E2YL5-min.jpg b/images/memes/cute-dog-with-cupcake-P9E2YL5-min.jpg similarity index 100% rename from images/cute-dog-with-cupcake-P9E2YL5-min.jpg rename to images/memes/cute-dog-with-cupcake-P9E2YL5-min.jpg diff --git a/images/embarassed-dog-on-bed-SA2BDZW.jpg b/images/memes/embarassed-dog-on-bed-SA2BDZW.jpg similarity index 100% rename from images/embarassed-dog-on-bed-SA2BDZW.jpg rename to images/memes/embarassed-dog-on-bed-SA2BDZW.jpg diff --git a/images/halloween-Pumpkin-min.jpg b/images/memes/halloween-Pumpkin-min.jpg similarity index 100% rename from images/halloween-Pumpkin-min.jpg rename to images/memes/halloween-Pumpkin-min.jpg diff --git a/images/happy-young-couple-eat-breakfast-in-bed-in-morning-RH4KQ72.jpg b/images/memes/happy-young-couple-eat-breakfast-in-bed-in-morning-RH4KQ72.jpg similarity index 100% rename from images/happy-young-couple-eat-breakfast-in-bed-in-morning-RH4KQ72.jpg rename to images/memes/happy-young-couple-eat-breakfast-in-bed-in-morning-RH4KQ72.jpg diff --git a/images/i-love-you-note-in-the-valentine-day-settings-X87BZ44.jpg b/images/memes/i-love-you-note-in-the-valentine-day-settings-X87BZ44.jpg similarity index 100% rename from images/i-love-you-note-in-the-valentine-day-settings-X87BZ44.jpg rename to images/memes/i-love-you-note-in-the-valentine-day-settings-X87BZ44.jpg diff --git a/images/love-from-the-past-PPBEUVU-min.jpg b/images/memes/love-from-the-past-PPBEUVU-min.jpg similarity index 100% rename from images/love-from-the-past-PPBEUVU-min.jpg rename to images/memes/love-from-the-past-PPBEUVU-min.jpg diff --git a/images/mexican-food-concept-EXFWKZG.jpg b/images/memes/mexican-food-concept-EXFWKZG.jpg similarity index 100% rename from images/mexican-food-concept-EXFWKZG.jpg rename to images/memes/mexican-food-concept-EXFWKZG.jpg diff --git a/images/portrait-of-happy-birthday-boy-B8VU4LZ.jpg b/images/memes/portrait-of-happy-birthday-boy-B8VU4LZ.jpg similarity index 100% rename from images/portrait-of-happy-birthday-boy-B8VU4LZ.jpg rename to images/memes/portrait-of-happy-birthday-boy-B8VU4LZ.jpg diff --git a/images/stonks-meme.jpg b/images/memes/stonks-meme.jpg similarity index 100% rename from images/stonks-meme.jpg rename to images/memes/stonks-meme.jpg diff --git a/images/streamlit-screenshot.png b/images/streamlit-screenshot.png new file mode 100644 index 0000000..507b866 Binary files /dev/null and b/images/streamlit-screenshot.png differ diff --git a/index-Copy1.ipynb b/index-Copy1.ipynb new file mode 100644 index 0000000..7709105 --- /dev/null +++ b/index-Copy1.ipynb @@ -0,0 +1,334 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "from nbdev.showdoc import *" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# memery\n", + "\n", + "> Search over large image datasets with natural language and computer vision!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "*^ that is so nerdy. can we do something like \"Local-first AI image search\"? or is that still too opaque*\n", + "\n", + "![meme about having too many memes](images/E2GoeMyWEAAkcLz.jpeg)\n", + "\n", + "The problem: you have a huge folder of images. Memes, screenshots, datasets, product photos, inspo albums, anything. You know that somewhere in that folder is the exact image you want, but you can't remember the filename or what day you saved it. There's nothing you can do buit scroll through the folder, skimming hundreds of thumbnails, hoping you don't accidentally miss it, hoping you'll recognize it when you do see it. \n", + "\n", + "Humans do this amazingly well. But even with computers, local image search is still a manual effort - you're still sorting through folders of images, like an archivist of old.\n", + "\n", + "**Until memery**.\n", + "\n", + "The `memery` package provides natural language search over local images. You can use it to search for things like \"a line drawing of a woman facing to the left\" and get _reasonably good results!_ \n", + "\n", + "You can do this over thousands of images (it's not optimized for performance yet, but search times scale well under O(n)). \n", + "\n", + "You can view the images in a browser GUI, or pipe them through command line tools. \n", + "\n", + "You can use `memery` or its modules in Jupyter notebooks, including GUI functions! \n", + "\n", + "Under the hood, `memery` makes use of **CLIP**, the [Contrastive Language-Image Pretraining transformer](https://github.com/openai/CLIP), released by OpenAI in 2021. CLIP trains a vision transformer and a language transformer to find the same latent space for images and their captions. This makes it perfect for the purpose of natural language image search. CLIP is a giant achievement, and `memery` stands on its shoulders." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Outline:\n", + "- Usage\n", + " - Install locally\n", + " - Use GUI\n", + " - Use CLI\n", + " - Use in Jupyter\n", + " - Use the library\n", + "- Development\n", + " - Notebook-driven development\n", + " - Pull the repo\n", + " - Branch and install\n", + " - Notebook-driven development\n", + " - Change the notebooks\n", + " - Test the notebooks\n", + " - Notebook-driven development\n", + " - Tangle the source code\n", + " - Weave the documentation\n", + "- Contributing\n", + " - Who works on this project\n", + " - How you can help\n", + " - What we don't do\n", + " - Thanks" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Install\n", + "\n", + "The necessary CLIP and torch packages will be installed by pip.\n", + "\n", + "*what does this mean? why are you telling me this? what if they aren't?*\n", + "\n", + "You might want to make sure you have a sane CUDA environment before and after this step if you're trying to use GPU. If you don't have a GPU, `memery` should still work on your CPU.\n", + "\n", + "*this is backwards and also impenetrable if you're not a nerd*\n", + "\n", + "If you have any trouble please **open an issue on Github**! I want to make this package useful for as many people as possible. Help me know what's going wrong :) \n", + "\n", + "*still true, pretty clear. use emoji or go home though*\n", + "\n", + "*This is the crucial command, huh? Seems a little far down the page. Maybe a quick-start section earlier, then a more in-depth Installation section here*\n", + "\n", + "`pip install memery`\n", + "\n", + "If you don't have a GPU for PyTorch, this command might help:\n", + "`pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html`\n", + "\n", + "*did you scrape this from stackoverflow? or the CLIP source code or what. it looks very specific and arbitrary. I think it's meant to install pytorch CPU version, but the average user shouldn't have to know that! if you are trying to use CUDA and GPU it should be an easy toggle, and if you're a regular person wihtout a data science project you shouldn't have to go through a bunch of weird command line stuff.*\n", + "\n", + "*In fact, how can we package this in a way that you don't have to type a single command? There has to be ways to wrap up a whole python setup and install it on a different computer, with all the same dependencies. Right? Ugh, unfortunately this is Python, so it might not exist. But I can at least scale the difficulty level with the user story.*" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "*Yes, user stories. We have to think about the interface between human and computer as another API, and look for the endpoints we can serve to different users with different protocols.*\n", + "\n", + "*This means thinking about the reasons people come to the program, and how they expect it to respond. Currently I see three user stories:* \n", + " * *i have images and want to search them with a GUI app*\n", + " * *streamlit GUI*\n", + " * *i have a program/workflow and want to use image search as one part of it*\n", + " * *CLI tool*\n", + " * *python module*\n", + " * *jupyter GUI*\n", + " * *i want to improve on and/or contribute to memery development*\n", + " * *clone the repo*\n", + " \n", + "\n", + "*Each one has an increasing level of complexity and expected knowledge.*" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Use GUI\n", + "\n", + "The browser GUI is a Streamlit app. You can run it from the command line with \n", + "\n", + "`memery serve`\n", + "\n", + "or set up a desktop shortcut to use it from your menu.\n", + "\n", + "\n", + "*Why do I have to use a CLI to get a GUI? This is very annoying. In any case, this does not provide instructuions on how to use the GUI or any of its particular quirks. There should be a screenshot, also*\n", + "\n", + "If you're in a Jupyter environment, you can summon an ipywidgets GUI directly into an output cell like this:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from memery.gui import appPage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "4a48874b5c9a4d05a85a04551e8cf5b0", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "AppLayout(children=(Box(children=(Text(value='images/', layout=Layout(max_width='80%'), placeholder='path/to/i…" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "app = appPage()\n", + "display(app)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "*This doesn't transfer to the README, so it should be fixed or be scrapped. Screenshots would work fine. Also I'm not sure if the interactive widgets would make sense without a kernel behind them anyway.*" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Use CLI\n", + "\n", + "From the command line, you can use `memery` on any folder and it will search for images recursively, returning a list object to stdout.\n", + "\n", + "*Why is it that behavior? Is it possible to make it not recursive? What is the usual behavior of a search in POSIX? What is the use case for a CLI memery: is it shell scripts? in-console image browsing? risk-tolerant batch modification? Think about this further*\n", + "\n", + "*at least you can control how much output it spews to your terminal*\n", + "\n", + "Pass the --n flag to control how many images are returned (default 10).\n", + "\n", + "`memery recall PATH/TO/IMAGE/FOLDER 'query' --n 20\n", + "`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Use as a library" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "*the following paragrah cannot be comprehended by any human mind. if you can even read to the end of it you should go touch grass*\n", + "\n", + "Simply use `queryFlow` to search over a folder recursively! The folder will be indexed, if an index doesn't already exist. Then any new images will be CLIP-encoded, an Annoy treemap built, and a list of ranked filenames returned.\n", + "\n", + "*okay, we survived. What are we supposed to say here instead?*\n", + "\n", + "*I think what he's trying to get at is explaining the underlying mechanism of the search. Really all we need here is a function signature, as anyone who's going to use it as a library will probably also be looking at the docs for further information. Have a link here too, that goes to the `core` file where the main flows and executors are explained.*" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from memery.core import queryFlow\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ranked = queryFlow('./images', 'dad joke')\n", + "\n", + "print(ranked[:5])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "*Okay, now we need a whole part about development." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "*Compile this notebook*" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Converted 00_core.ipynb.\n", + "Converted 01_loader.ipynb.\n", + "Converted 02_crafter.ipynb.\n", + "Converted 03_encoder.ipynb.\n", + "Converted 04_indexer.ipynb.\n", + "Converted 05_ranker.ipynb.\n", + "Converted 06_fileutils.ipynb.\n", + "Converted 07_cli.ipynb.\n", + "Converted 08_jupyter_gui.ipynb.\n", + "Converted 09_streamlit_app.ipynb.\n", + "Converted index.ipynb.\n" + ] + } + ], + "source": [ + "from nbdev.export import notebook2script; notebook2script()\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/index.ipynb b/index.ipynb index 6b72804..9291f9c 100644 --- a/index.ipynb +++ b/index.ipynb @@ -75,7 +75,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Install\n", + "## Installation\n", "\n", "With Python 3.6 or greater:\n", "`pip install memery`\n", @@ -84,14 +84,16 @@ "\n", "`pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html`\n", "\n", - "Someday memery will be packaged in an easy to use format, but since this is a Python project it is hard to predict when that day will be." + "Someday memery will be packaged in an easy to use format, but since this is a Python project it is hard to predict when that day will be.\n", + "\n", + "If you want to help develop memery, you'll need to clone the repo. See below." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## How to use\n", + "## Usage\n", "\n", "What's your use case? \n", "\n", @@ -311,33 +313,143 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "---\n", + "Here's the first result from that list:\n", + "\n", + "![](images/memes/Wholesome-Meme-68.jpg)\n", "\n", - "*Okay, now we need a whole part about development." + "\n", + "So that's how to use memery. Let's look at how you can help make it better." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "---\n", + "## Development\n", "\n", - "*Compile this notebook*" + "Memery is a different beast than most pieces of code you've seen. It's a *literate program*: a program written for human beings to read.\n", + "\n", + "Nothing in this code is particularly special. The algorithms, data structures, and pipeline are either bog-standard or even subpar. The model was developed by OpenAI, and the tree indexer by Spotify. All I did was glue a bunch of disparate things together. The only reason nobody did it before me is that there's no money in it.\n" ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], - "source": [] + "source": [ + "### Notebook-driven development\n", + "\n", + "The thing that makes this program interesting is that it was developed *in Jupyter notebooks*. Each component has its code and documentation in the same place, a `.ipynb` notebook.\n", + "\n", + "This is possible thanks to a new-ish project called `nbdev`. It provides literate programming functionality for notebooks. Specifically, it allows the programmer to automatically weave code and tangle documentation from the content of the notebooks!\n", + "\n", + "This means that relevant docs, code and tests for that code all live in the same location. And ideally, that place is a notebook *written for humans*. I'm still getting the hang of this, but it means explaining *why* something works, rather than simply how to do it.\n", + "\n", + "So that's the *why* of notebook-driven development. Let's look at *how* that works in practice." + ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], - "source": [] + "source": [ + "### Pull the repo\n", + "\n", + "Clone this repository from Github:\n", + "\n", + "`git clone https://github.com/deepfates/memery.git`\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install dependencies\n", + "Enter the `memery` folder and install requirements:\n", + "\n", + "```\n", + "cd memery\n", + "pip install requirements.txt`\n", + "```\n", + "\n", + "Feel free to use a virtual environment manager of your choosing. I ought to do this but I've been lucky too long and it's gone to my head.\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Branch the code\n", + "\n", + "\"Small branches, quickly merged\" should be the motto here. Another tragic case of \"do what I say, not what I do\".\n", + "\n", + "For example, in updating the literate documentation for memery I created a branch called `literate`:\n", + "\n", + "`git checkout -b literate`\n", + "\n", + "Within this branch, I can work on different files and subtasks. I can commit several changes in a row, and double check all my tests, before trying to merge with the upstream `main` repo through a pull request." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Notebook-driven development\n", + "\n", + "To test memery as you work on it, you'll want to install it locally. This is as simple as using `pip` with \"editable\" enabled:\n", + "\n", + "`pip install -e .`\n", + "\n", + "The `.` refers to your local working directory; if this command doesn't work, try replacing that character with the full path of your memery repo.\n", + "\n", + "Now, within the main memery folder there is a subfolder `memery/memery`. This contains `.py` files, which are the source code for the repo. You might be tempted to edit these Python files directly, but you must hesitate.\n", + "\n", + "Remember: we are doing notebook-driven development. The Python files" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Change the notebooks" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Test the notebooks" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notebook-driven development" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Tangle the source code" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Weave the documentation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "*Compile this notebook*" + ] }, { "cell_type": "code", diff --git a/settings.ini b/settings.ini index 7bb94a7..f80bb6f 100644 --- a/settings.ini +++ b/settings.ini @@ -24,4 +24,4 @@ doc_baseurl = /memery/ git_url = https://github.com/deepfates/memery/tree/main/ lib_path = memery title = memery -requirements = annoy torchvision==0.8.2 tqdm ipywidgets torch==1.7.1 ipython Pillow typer streamlit clip-by-openai +requirements = annoy torchvision==0.8.2 tqdm ipywidgets torch==1.7.1 ipython Pillow typer streamlit clip-by-openai nbdev==1.1.19 fastcore==1.3.21 nbconvert==5.6.1 \ No newline at end of file