Zero vectors can not be normalized

[gilbertwat]

Hi there!

I am using Rake to try to automate my Jekyll blog generation and find such errors. I didn't encounter such when using jekyll serve --lsi. Any pointers? 😄

Rebuilding index... rake aborted!
Zero vectors can not be normalized
/Library/Ruby/Gems/2.0.0/gems/classifier-reborn-2.0.4/lib/classifier-reborn/lsi.rb:143:in `block in build_index'
/Library/Ruby/Gems/2.0.0/gems/classifier-reborn-2.0.4/lib/classifier-reborn/lsi.rb:141:in `times'
/Library/Ruby/Gems/2.0.0/gems/classifier-reborn-2.0.4/lib/classifier-reborn/lsi.rb:141:in `build_index'
/Library/Ruby/Gems/2.0.0/gems/jekyll-3.0.1/lib/jekyll/related_posts.rb:38:in `build_index'
/Library/Ruby/Gems/2.0.0/gems/jekyll-3.0.1/lib/jekyll/related_posts.rb:20:in `build'
/Library/Ruby/Gems/2.0.0/gems/jekyll-3.0.1/lib/jekyll/document.rb:455:in `related_posts'
/Library/Ruby/Gems/2.0.0/gems/jekyll-3.0.1/lib/jekyll/renderer.rb:41:in `run'
/Library/Ruby/Gems/2.0.0/gems/jekyll-lunr-js-search-3.0.0/lib/jekyll_lunr_js_search/page_renderer.rb:17:in `prepare'
/Library/Ruby/Gems/2.0.0/gems/jekyll-lunr-js-search-3.0.0/lib/jekyll_lunr_js_search/page_renderer.rb:34:in `render'
/Library/Ruby/Gems/2.0.0/gems/jekyll-lunr-js-search-3.0.0/lib/jekyll_lunr_js_search/search_entry.rb:20:in `create'
/Library/Ruby/Gems/2.0.0/gems/jekyll-lunr-js-search-3.0.0/lib/jekyll_lunr_js_search/indexer.rb:64:in `block in generate'
/Library/Ruby/Gems/2.0.0/gems/jekyll-lunr-js-search-3.0.0/lib/jekyll_lunr_js_search/indexer.rb:63:in `each'
/Library/Ruby/Gems/2.0.0/gems/jekyll-lunr-js-search-3.0.0/lib/jekyll_lunr_js_search/indexer.rb:63:in `each_with_index'
/Library/Ruby/Gems/2.0.0/gems/jekyll-lunr-js-search-3.0.0/lib/jekyll_lunr_js_search/indexer.rb:63:in `generate'
/Library/Ruby/Gems/2.0.0/gems/jekyll-3.0.1/lib/jekyll/site.rb:154:in `block in generate'
/Library/Ruby/Gems/2.0.0/gems/jekyll-3.0.1/lib/jekyll/site.rb:153:in `each'
/Library/Ruby/Gems/2.0.0/gems/jekyll-3.0.1/lib/jekyll/site.rb:153:in `generate'
/Library/Ruby/Gems/2.0.0/gems/jekyll-3.0.1/lib/jekyll/site.rb:58:in `process'

[parkr]

Not sure! Do you have any posts which aren't like the others? Perhaps one without any content?

[Ch4s3]

This is one of those tricky Matrix building errors, I think. Any empty post might cause this.

[chris357]

Just started playing with this today, and I won't pretend to understand Vector::ZeroVectorError: Zero vectors can not be normalized in the context of this gem. But I was able to replicate the issue by editing the Readme example.

No issue:

require 'classifier-reborn'
lsi = ClassifierReborn::LSI.new
strings = [ ["This text deals with dogs. Dogs.", :dog],
            ["This text involves dogs too. Dogs! ", :dog],
            ["This text revolves around cats. Cats.", :cat],
            ["This text also involves cats. Cats!", :cat],
            ["This text involves birds. Birds.",:bird ]]
strings.each {|x| lsi.add_item x.first, x.last}

p lsi.classify "This text is also about dogs!"

Note I'm going to change the first string.

No issue:
["This text deals with dogs.", :dog]

Still no issue:
["This text deals.", :dog]

Issue:
["This te.", :dog]
=>Vector::ZeroVectorError: Zero vectors can not be normalized

Not sure if thats helpful.
(Running this as a rake task in a Rails app)

[Ch4s3]

@chris357 really small inputs without meaningful words i.e. only stop words are known to break LSI. I'm looking into how to handle this more gracefully.

[fabriziofortino]

+1 same error here

[Ch4s3]

fixed by #77

[ArtOfCode-]

Time for some reanimation of zombie threads... I ran into this issue again.

Classifier Reborn 2.1.0, GSL 2.1.0.3. I'm taking data from a Rails app and trying to feed it into an LSI classifier:

Post.where.not(body: nil).each do |p|
  body = p.body.tr "\n", ''
  if p.is_tp
    lsi.add_item p.body, :spam
  elsif p.is_fp
    lsi.add_item p.body, :ham
  end
end

The data being fed in looks something like this:

<p>I am looking for either a web app or installable program that will track themovies I have watched.</p><p>I have found a few online but I would like one that also tracks how many times Ihave watched each movie, as I like to rewatch many of my movies.</p><p>I would also like to be able to easily sort the data, for example sorting by"last watched" or sorting by number of views.</p><p>Other requirements</p><ul><li>if installable program, must work with Windows</li><li>no answers saying "use a spreadsheet"</li><li>no answers saying "make your own"</li><li>no Windows Media Player (does not support MKV)</li><li>no Banshee (Windows version is out of date)</li></ul>

Getting the same old ZeroVectorError: Zero vectors can not be normalized. Stripping the HTML out doesn't help.

[Ch4s3]

@ArtOfCode- can you try our master branch? There's been a lot of work dones since 2.1.0 was released.

[ArtOfCode-]

Sure thing. I'm not at a dev machine right now, but I'll give it a shot when I get back later on.