MNIST example

[breznak]

• this is work from @ctrl-z-9000-times , I was playing with the branch and wanted to help, get it in faster.

• merged recent master

• reverted changes to SP, this PR aims only to add MNIST; revert 3193892 to continue where you left off.

• bundle (how?) MNIST data

• the example now segfaults

• optimize parameters for better score! (currently 11% 30% 43%)

[ctrl-z-9000-times]

@breznak,
It appears you started from behind the HEAD. I think a better work flow would have been to:

1. Start from the latest MNIST branch,
2. Verify that it works (no segfault, 95% accuracy)
3. Make new branch off of MNIST
4. Revert the SpatialPooler stuff to master branch by git checkout master SpatialPooler

bundle (how?) MNIST data

We need download this. Uncompressed, the dataset is 53 Megabytes so we should not check it into git! The dataset has an authoritative source on the internet. A python script can download, unzip and uncompress the files in a cross platform way. I think that numenta has an example downloader script.

Data source: http://yann.lecun.com/exdb/mnist/

[breznak]

my comments about the PR

[breznak]

A python script can download, unzip and uncompress the files in a cross

platform way. I think that numenta has an example downloader script. I think even cmake will be able to download and extract it.

official sources

Yes, but I wonder if the csv wouldn’t be a better format? It’s not that this is some critical data, and we can verify the content matches the original sources, cmake can verify checksums for us.

…

-- Marek Otahal :o)

[dkeeney]

I think even cmake will be able to download and extract it.

Yes, quite simple.
The question is do we really want it as part of the unit tests that a build every time we submit? That is going to take a lot clock time to run.

This is a great tool to confirm that we have good results but should it be in the CI?

[breznak]

This is a great tool to confirm that we have good results but should it

be in the CI? So far it’s an example, so it doesn’t have to be a test. Although it runs only 40s , not that long. Or we could later do performance tests that run nightly, as Matt mentioned today

…

-- Marek Otahal :o)

[breznak]

Finally the PR is ready.

but hopefully I can get a solid 80% with global inhibition.

apparently, global inhibition is not suitable for 2D images. Could you please retry the parameter optimization some time with local inhibition? It'll take a long time, but should perform no worse than the proposed marcoloumn-aware inhibition.

Otherwise, please review as is.

[dkeeney]

Please turn back on shared library.

[dkeeney]

Thanks :) I spent a week getting shared lib to work so I am a little attached to it.

[breznak]

@ctrl-z-9000-times I am still wondering why the accuracy is so not-good :) with local inhibition (low perf with global inh is "intuitive") compared to your "extended SP". If I understand it, the clever idea with MacroCol SP/inh is that you use the n+1-th dimension to force X cells to cover the (subsets) of the same portion of input space/potential pool.
So the results here with local inh suck just because the low number of columns (variations of input coverage?) (which is due to limited speed)

[ctrl-z-9000-times]

the clever idea with MacroCol SP/inh is that you use the n+1-th dimension to force X cells to cover the (subsets) of the same portion of input space/potential pool.

That's half of it. The other half is to use global inhibition over the N+1-th dimension, and treat everything in the first N dimensions as a totally separate inhibition areas. The clever trick here is to treat every pair of cells which could possibly interact as either:

• very close to each other, use global inhibition within a macro-column or
• very far away from each other, too far away to interact with each other.

[breznak]

Bump, please review

[dkeeney]

Looks good. 👍

[Thanh-Binh]

@ctrl-z-9000-times you have your implementation GridCell as an extension of SP. I think it is also useful for Mnist classification. Any comment from you?