ValueError: user_items must contain 1 row for every user in userids

Hi ,I use implicit , the user and item count is enough ，and the model has build success，and invoke model similiar_items method can get real answer ,

>>> related = model.similar_items(item_lable, N=10)
>>> related
(array([   2,  988, 2141, 5690, 3721, 5521, 5401, 5408, 3808,  310],
      dtype=int32), array([1.0000001 , 0.92822003, 0.77317417, 0.7726441 , 0.7664236 ,
       0.765062  , 0.7645345 , 0.7614211 , 0.75764   , 0.7574534 ],
      dtype=float32))

but when I invoke model.recommend( ) thell the error

>>> data = {
...     'model.item_factors': model.item_factors,
...     'model.user_factors': model.user_factors,
...     'item_labels': item_idx,
... }
>>> model.item_factors = data['model.item_factors']
>>> model.user_factors = data['model.user_factors']
>>> model._YtY = model.item_factors.T.dot(model.item_factors)
>>> item_labels = data['item_labels']
>>> user_idx
Int64Index([101985756, 184594004, 175219573, 185595506, 190138165, 189930980,
            219915716, 170969740, 170672951, 191737873,
            ...
            211615466, 170577779, 100900641, 104947861, 100570017, 177317703,
            100334831, 104988856, 170053121, 104510246],
           dtype='int64', length=5728)

>>> user_id = 175219573
>>> user_label = list(user_idx).index(user_id)
>>> sparse_user_items = sparse_item_user.T.tocsr()
>>> recommendations = model.recommend(user_label, sparse_user_items)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/muller/opt/anaconda3/envs/helenenv/lib/python3.7/site-packages/implicit/cpu/matrix_factorization_base.py", line 45, in recommend
    raise ValueError("user_items must contain 1 row for every user in userids")
ValueError: user_items must contain 1 row for every user in userids

>>> sparse_user_items
<5728x5223 sparse matrix of type '<class 'numpy.float64'>'
	with 156164 stored elements in Compressed Sparse Row format>

>>> sparse_item_user
<5223x5728 sparse matrix of type '<class 'numpy.float64'>'
	with 156164 stored elements in Compressed Sparse Row format>

>>> sparse_item_user.shape
(5223, 5728)

>>> sparse_user_items.shape
(5728, 5223)

as you see the user_items matrix is not null , I do not know why ,is some user_ids is not exist? or some user_ids has not data row in user_item

There have been a couple of API changes in the last release - #481 has the high level overview.

The API change that is causing your issue is #526 . The idea here was that we don't need empty rows in the user_items matrix for users that aren't being recommended - so we need to update invocations to be something like model.recommend(userids, user_items[userids]) instead of model.recommend(userids, user_items) . This change means that if you're recommending items for a single userid=10000000 - you don't need to pass in a sparse matrix with a 10000001 entries in the indptr member , and will be slightly faster in this case.

The fix here is to go model.recommend(user_label, sparse_user_items[user_label]) instead of model.recommend(user_label, sparse_user_items)

very thanks ,solve the problem

A thought after having the same confusion and coming here to resolve it:

The error message ValueError: user_items must contain 1 row for every user in userids is slightly confusing because the matrix sparse_user_items does contain 1 row for every user in userids. The problem is that it includes other rows too.

Perhaps there are really two different errors to potentially raise here:

ValueError: user_items must contain 1 row for every user in userids
ValueError: user_items must not contain rows for user not in userids
The error encountered here being the second one.

@benfred

@benfred : Have we made some more changes to the function? I was able to score (with the above changes) till 15th Dec'22.
The following code was executed :

sparse_item_user = sparse.csr_matrix((df_model['cnt_clk'].astype(float), (df_model['content_id'], df_model['sub_id'])))
sparse_user_item = sparse.csr_matrix((df_model['cnt_clk'].astype(float), (df_model['sub_id'], df_model['content_id'])))

sparse_user_item
<4504602x82354 sparse matrix of type '<class 'numpy.float64'>'
	with 50534476 stored elements in Compressed Sparse Row format>

sparse_item_user
<82354x4504602 sparse matrix of type '<class 'numpy.float64'>'
	with 50534476 stored elements in Compressed Sparse Row format>

if __name__ == '__main__':
    content = []
    scores = []
    subs = []
    def performRecommendations(i):
        recommended = model.recommend(subids_test[i], sparse_user_item[i], 10)
        
        subs.append(subids_test[i])
        for (c, s) in zip(recommended[0], recommended[1]):
            content.append(c)
            scores.append(s)
            #subs_1.append(subids_test[i])
            #print("c:",c, "s:", s)

##Above block of code is for scoring
        
    def doRecommendations(number, start):
        newList = [performRecommendations(i) for i in range(start, number)]

##Below block of code is for parallel processing 

    th_count = 62
    th_list = []
    begintime=time.time()

    for th in range(1, th_count+1):
        q, mod = divmod(len(subids_test), th_count)
        number = th * q
        start = number - q
        if mod != 0 and th == th_count:
            number = number + mod
        thread = Thread(target=doRecommendations, args=(number, start,))
        th_list.append(thread)
    for th in th_list:
        th.start()
    for th in th_list:
        th.join()
    endtime = time.time()
    print("The total time taken:",endtime-begintime)

Now when I am trying to score again, I am getting the following error:

Exception in thread Thread-6:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "<ipython-input-54-0958b4b3cf6d>", line 17, in doRecommendations
    newList = [performRecommendations(i) for i in range(start, number)]
  File "<ipython-input-54-0958b4b3cf6d>", line 17, in <listcomp>
    newList = [performRecommendations(i) for i in range(start, number)]
  File "<ipython-input-54-0958b4b3cf6d>", line 6, in performRecommendations
    recommended = model.recommend(subids_test[i], sparse_user_item[i], 10)
  File "implicit/recommender_base.pyx", line 171, in implicit.recommender_base.MatrixFactorizationBase.recommend
  File "implicit/recommender_base.pyx", line 321, in implicit.recommender_base.MatrixFactorizationBase._user_factor
IndexError: index 360503 is out of bounds for axis 0 with size 82354

There have been a couple of API changes in the last release - #481 has the high level overview.

The API change that is causing your issue is #526 . The idea here was that we don't need empty rows in the user_items matrix for users that aren't being recommended - so we need to update invocations to be something like model.recommend(userids, user_items[userids]) instead of model.recommend(userids, user_items) . This change means that if you're recommending items for a single userid=10000000 - you don't need to pass in a sparse matrix with a 10000001 entries in the indptr member , and will be slightly faster in this case.

The fix here is to go model.recommend(user_label, sparse_user_items[user_label]) instead of model.recommend(user_label, sparse_user_items)

Could this be reflected in the Quickstart section? For whoever follows the basic instructions, this is likely to happen often, right? Thank you!

hi, what should i do if i still in trouble with this and it told me index 945 is out of bounds for axis 1 with size 240.
model1.recommend(0,user_item_sparse1[0])
i used something like this.

model.recommend(user_label, sparse_user_items[user_label])

Getting below error while using above function

IndexError Traceback (most recent call last)
Cell In[33], line 1
----> 1 model.recommend(user_label, sparse_user_items[user_label])

File ~\anaconda3\Lib\site-packages\implicit\cpu\matrix_factorization_base.py:79, in MatrixFactorizationBase.recommend(self, userid, user_items, N, filter_already_liked_items, filter_items, recalculate_user, items)
76 if items is not None:
77 filter_query_items = _filter_items_from_sparse_matrix(items, filter_query_items)
---> 79 ids, scores = topk(
80 item_factors,
81 user,
82 N,
83 filter_query_items=filter_query_items,
84 filter_items=filter_items,
85 num_threads=self.num_threads,
86 )
88 if np.isscalar(userid):
89 ids, scores = ids[0], scores[0]

File topk.pyx:32, in implicit.cpu.topk.topk()

File topk.pyx:54, in implicit.cpu.topk._topk_batch()

IndexError: index 977 is out of bounds for axis 1 with size 944

Navigation Menu

Repository navigation

ValueError: user_items must contain 1 row for every user in userids #535

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

ValueError: user_items must contain 1 row for every user in userids #535

Description

Activity

benfred commented on Jan 29, 2022

mullerhai commented on Jan 29, 2022

lanejere5 commented on Apr 20, 2022

ritesh1187 commented on Jan 2, 2023

jaskirat8 commented on Feb 27, 2023

sejkko commented on Apr 14, 2023

llkk2021 commented on May 16, 2023

tangbufanwei commented on Jan 18, 2024

abhinav221294 commented on Feb 22, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions