Skip to content

How to compute rank_items for all my users?  #384

Closed

Description

@ManuelRios18

Hello,

I would like to know the best way to rank a handful of items for all my users, I currently have a huge number of users (around 5 million different users) and I need find the fastest way to rank 200 items to all of them.

Which is the best way to do this ?
calling the method rank_items() for each user sounds like a bad idea, I guess that there is a better factorized way to do this.

Can someone give me an advice ?

Thanks in advance.

Activity

  1. mohjeih commented on Aug 4, 2020

    @mohjeih

    Hello,

    One solution could be implementing your own rank_item_all method with a custom C++ function filling numpy arrays in parallel, similar to recommend_all method.

  2. ManuelRios18 commented on Aug 4, 2020

    @ManuelRios18
    Author

    Hello,

    One solution could be implementing your own rank_item_all method with a custom C++ function filling numpy arrays in parallel, similar to recommend_all method.

    @mohjeih
    Where can I get the recommend_all method written in C++ ? And where is the current "rank_items" method implemented ?
    In order to have a guideline

  3. mohjeih commented on Aug 4, 2020

    @mohjeih

    @ManuelRios18 Sorry for the confusion. recommend_all and rank_items are implemented in recommender_base.pyx (Cythonized module). Please see here .recommend_all sorts array of scores in parallel using a C++ sorting function, fargsort_c, which is implemented in topnc.cpp. The wrapper to this sorting function is defined at the beginning of recommender_base.pyx.

  4. marinkreso95 commented on Apr 9, 2021

    @marinkreso95

    Hi
    I have the same problem as @ManuelRios18 (if you found a solution for this, please share)
    Here is a code snippet from my project. Does somebody have another way to do this properly? Because this solution is taking a lot of time

        row = y.recipe_id_comb.astype(item_c).cat.codes
        col = y.user_id.astype(user_c).cat.codes
        
        item_user_matrix = csr_matrix((y["rating"], (row, col)), shape=(item_c.categories.size, user_c.categories.size))
        user_item_matrix = csr_matrix((y["rating"], (col, row)), shape=(user_c.categories.size, item_c.categories.size))
        
        # Train MF model
        mf_model = implicit.als.AlternatingLeastSquares(factors=opt_fac, regularization=opt_reg, iterations=opt_itr)
        mf_model.fit(item_user_matrix)
    
        # Extract outputs from MF model
        score_mf = row.copy()
        for ind_i in np.unique(col):
            mf_y = mf_model.rank_items(ind_i, user_item_matrix, row[col == ind_i].astype(int))
            score_mf[col == ind_i] = score_mf[col == ind_i].replace(dict(mf_y))
        score_mf = score_mf.to_numpy().reshape((len(score_mf), 1))
    
  5. ManuelRios18 commented on Apr 10, 2021

    @ManuelRios18
    Author

    Hello, Sadly I couldn't find an efficient solution, what I ended up doing was simply multiply user and item embeddings, load those results into my company's Landing Zone, and then I ranked the items using basic SQL queries.

    Happy coding!

    @marinkreso95

  6. benfred commented on Jan 3, 2022

    @benfred
    Owner

    @ManuelRios18 the latest main branch has support for this (though does have some breaking API changes ). see #481

    model = implicit.als.AlternatingLeastSquares()
    model.fit(user_items)
    
    # rank the first 200 items in the model for all the users
    userids = np.arange(model.user_factors.shape[0])
    items = np.arange(200)
    ids, scores = model.recommend(userids, items=items)
  7. linked a pull request that will close this issueApi refactor #494on Jan 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions