Reader: Add consistent search Tracks events and traintracks events

[gibrown]

Reader: Add consistent search Tracks events and traintracks events

Now that search is launched on all three platforms there are a few places where the stats we are tracking do not exactly match each other. Here are the set of Tracks events in Calypso that I think we should also implement on mobile (with the appropriate name changes):

• calypso_reader_search_loaded: wordpress.com/read/search is loaded with an empty search string
• calypso_reader_search_performed: a search request has been sent to the API. The web does search as you type with the search getting sent once there is a 500ms delay, but I think there is no reason the mobile apps wouldn't work with a similar event.
• calypso_reader_searchcard_clicked: User clicks on a result to do an article open.
• calypso_traintracks_render (example props: fetch_algo: read:search:posts/1, fetch_blog_id: 123, fetch_post_id:123, query: news): a search result is rendered for the user to see. This means that if the client requests 10 results, then when it gets those results to display (and ideally when it actually shows them to the user) it will fire 10 separate events. See "traintracks" below.
• calypso_traintracks_interact (example props: action: follow, fetch_algo: read:search:posts/1, fetch_blog_id: 123, fetch_post_id:123, query: news): a search result is interacted with by the user (could also be in the full post view). List of interaction actions is: article_liked, article_opened, searchcard_clicked, site_followed, site_unfollowed. This is sort of a meta event that wraps up other Tracks events in a common way. See "traintrackss below".

Mobile apps also have some support for suggesting searches to users based on their past search history. It would be good to make sure we have tracks events for these and also pass through the query that the user selected. Maybe call it wpios_reader_search_suggest_selected and wpios_reader_search_suggest_shown?

What are TrainTracks?

We want a way to improve our algorithms (A/B test new ones) without having to change the clients. TrainTracks is a common method for tracking results from any api endpoint and we are deploying it for related posts, cold start, search, recommendations, etc

Key problem it solves is that a data scientist can develop and deploy an algorithm change and launch it to all platforms with no coordination. This should help is continually improve our algorithms over time. The idea is loosely inspired by this article: http://multithreaded.stitchfix.com/blog/2016/03/16/engineers-shouldnt-write-etl/ Make launching new algorithms routine and streamlined. Avoid having to sync up between the front end and back end. For instance, I ran an A/B test looking at whether to rank more recent content higher last week. And now we are running an A/B test where we let older content rank higher. I don't currently have a way to track these results on our mobile platforms because they aren't sending TrainTracks events.

So what is our framework (currently it may evolve as we find problems):

• When an algorithmic api responds to a request for results we return a "railcar" attached to each result (yes, we're taking this metaphor a little far ;) )
• Railcar contains the following fields:
  • fetch_alg - string identifying the algorithm that generated th
  • fetch_position - integer. The ranked position within the results
  • fetch_lang - the language(s) the algorithm was performed against
  • fetch_query - the user entered query
  • railcar - a unique ID that we use for matching render and interaction events
  • rec_blog_id - the blog id for this result
  • rec_post_id - the post id for this result
  • rec_feed_id - the feed id for this result
  • rec_feed_item_id - the feed item id for this result
  • more to be added in the future
• That's a lot of fields, but as a client of TrainTracks, all the client needs to do is take every property in the railcar and add it as an event property to the render and interact traintracks events
• The API also returns a top level "algorithm" field which is a string that should be used when paging the API. This ensures that we get the same algorithm when paging and so we won't see duplicate results.

What events trigger "Interaction" events?

Anything the user can do on result (open, like, comment, follow, etc). There is a list in Calypso here: https://github.com/Automattic/wp-calypso/blob/ca16ae870a6c8df1cb050936f40abb53fc1a2994/client/reader/stats.js#L93

These events then get truncated to remove the prefix and are added to the "action" property: https://github.com/Automattic/wp-calypso/blob/ca16ae870a6c8df1cb050936f40abb53fc1a2994/client/reader/stats.js#L107

cc @xyu

[gibrown]

Issue also in iOS: wordpress-mobile/WordPress-iOS#5817

[gibrown]

An additional point that just came up in Calypso is that the client should populate a property called 'ui_algo' to indicate where the rendered results are being displayed.

This is to differentiate between using the same api endpoint to populate data in two different UIs. In Calypso the current case we are dealing with is the search api being used at both '/recommendations/start' and '/read/search'. So we are using those two strings to indicate where in the UI the api results are being rendered and interacted with.