Add filters to Nearby

[misaochan]

This was inspired by #618 , but I decided to create a new issue as this is a much more general suggestion.

I wonder if it would be a good idea to have filters where people can filter out certain types of Nearby items. The most obvious usage would be for the "instanceOf" label, where someone could say they wanted "historic sites" or "suburbs" etc specifically, and only see those pins on the map. And of course, for the items that don't exist anymore, as per #618 . Maybe even filter for items that have or don't have Wikipedia articles, etc?

This would likely need to be done alongside #591 , as currently our range and location of pins is a bit restrictive.

[gupta1anubhav]

Even I was thinking of the same thing as the app shows places inside fix radius around user's location.
And random important places are shown.

[nicolas-raoul]

I would totally use this feature to filter out schools, to avoid the creepiness haha

When I go to a place where there are a lot of pins I might want to only show the important ones that are not closed. But once I have uploaded most of them and the pins are getting rare, I will probably want to show the other ones too.

What is important is probably subjective.

[misaochan]

I'm curious about the discussion of "important" - is this actually a Wikidata tag or are we just taking the lay meaning of it?

@nicolas-raoul Haha, good usage. ;) I've been thinking a bit about using the "instanceOf" label as a filter. There are many types of instanceOf, probably more than we can easily display in a checklist, I think. I wonder how we can work around that. Should we just allow filtering of several common ones? Should we have a "More" button that users can click to be brought to a different activity that shows all of them?

[nicolas-raoul]

There is no concept of "importance" in Wikidata. I just meant that some people want to take pictures of all religious buildings, while other people want to take pictures of all religious building, it is just a matter of personal interests.

I started computing statistics about the "instanceOf" of the points we deal with, it will probably take a day or two before all results get computed but here are the results so far:
class-statistics.txt
(only items with latitude/longitude and no image)

[nicolas-raoul]

Here are the types of items that have a latitude/longitude but do not have a picture:

 282548 mountain
 210804 human settlement
 165928 river
  94992 stream
  82413 village
  66246 hill
  44348 island
  34886 dam
  34651 movie theater
  34643 building
  31652 valley
  31146 unincorporated community
  25590 watercourse
  23946 lake
  23486 point
  23315 desa
  21930 fourth level administrative division in Indonesia
  21741 spring
  19563 town in China
  17906 canal
  17637 airport
  17610 park
  16816 wadi
  16611 hamlet
  16557 ravine
  16189 high school
  15103 radio station
  14727 lagoon
  13940 former building or structure
  12896 bight
  12389 rural commune of Vietnam
  12180 train station
  11878 ridge
  11632 house
  11104 township of the People's Republic of China
  10300 rural settlement of Russia
   9467 reservoir
   ...

Full file: stats2.txt
My computer crashed in the middle so there is no data for Europe/Africa. I will run my script again in a month, for now the data above gives an idea of what to expect.

As you can see:

• There is a very long tail. If we want to cover 75% of the data we have to provide 40 types.
• We can group types, for instance "river" and "stream". While we could get the SPARQL request to get parent classes, that would probably make the already slow request even slower. The pragmatic solution is probably to hard-code mappings from their QIDs to categories that make sense to our app, for instance Q4022 (river), Q47521 (stream) → "watercourse"

[misaochan]

@nicolas-raoul Thanks! The data is very interesting, I actually had no idea there were so many. It is also interesting that there are more "rural commune of Vietnam"s than there are global train station points, lol.

This does complicate things a bit. Maybe we should come up with a simple prototype first, and then see if many people actually use the filters, before enhancing it by hardcoding mappings to improve coverage?

My suggestion is that we take all of the ones with >10k points, and remove the country-specific ones. Then in the filter UI we have a checklist with "Doesn't exist anymore", "Has Wikipedia article", and "Instance of". If the user checks the Instance Of item, we take them to a separate fragment that has all the points I mentioned and they can choose which ones to check.

What do you guys think?

[VojtechDostal]

I'd LOVE to have a possibility of filtering the results (filtering out streets in my case, photographing them is BORING :-) ).

I was thinking that maybe we could use parts of the queries used by Petscan (in the Wikidata tab, see attached image). You can just type any number of properties or items separated by commas and they will get filtered. This is an option for power users anyway.

[neslihanturan]

Here is my proposed design for this filtering feature:

Actually I am quite happy with how design evolved. Here are my decision parameters:

• Since they were one of the suggested element in material design guidelines for filtering, I decided to use chips. The other alternative is checkboxes.
• The first field, state of the place has limited number of options and user input would be nonsense. So I decided to use filter chips with them which you can click to select/deselect.
• The second field will have almost 40 options, so I thought writing some inputs could help to choose. Thus, I decided to use input chips with this field.
• It would be great if we could design small new icons for each options, not necessary though.
• This design also provides user to see their selections on the same screen.

---

• My only concern is, is it hard to discover the input field on second option? However, I think people used to use this types of action, ie. gmail has the same.

Can't wait to hear your feedbacks:)

[VojtechDostal]

Hello Neslihan, it looks great from a UX perspective. I am worried that it's not customizable enough, though. Will it be possible to filter custom classes of objects? For example, Czech Republic has all streets imported and I'd like to filter them out. It could be the same for eg. buddhist temples in Nepal. Is it possible to enter any type of class or will your tool be limited to only 40 options?

[misaochan]

Hi @neslihanturan , thanks for the mockup!

• I love the "search" field for place type, good thinking. :) If user discovery is a concern, maybe have a "search" icon next to the field?
• Love the chips for place state.
• I think the entire filter card should be collapsible, similar to the cards used in the upload process. Some people are on small screens, and this card would take up a lot of unnecessary space if people aren't using it
• "Has photo" should probably be: "Has Wikipedia article"?
• I think checkboxes would suit place type better than chips. The reason is that people should be able to filter for more than one at a time, and also for "everything but". For instance, someone might want "everything except schools" whereas someone might want "only monuments", and another person might want "only schools and monuments", etc. So IMO with checkboxes, we can start with everything enabled, and then users can disable certain class types as they wish. There should also be options to "select all" and "deselect all". However I am not sure how to get this to work with the search field, it may require some reconfiguration.

@VojtechDostal we were discussing the issue of classes here. Edited: I looked over our current code and it does seem feasible backend-wise to allow users to search for custom classes (with the caveat that if they search wrongly, no pins will show up). However I'm not sure yet how we can implement this UX-wise. Perhaps it could be an additional "custom" option separate from the list of mainstream options?

[VojtechDostal]

Hmm, that's of little use to our community but I understand your reasons.

I originally envisioned an input box somewhere in settings which would permanently alter search results for the user for all his/her queries (unless they decide to change the settings). It could have worked like Petscan which I mentioned in June last year (see above).

[misaochan]

@VojtechDostal I revised my comment after looking through the code just now, I think it might be doable.

My suggestion for this: If the user types something in the search field that isn't one of our preset options (say, "street", although from the looks of Nicolas's stats.txt, that should be high enough in the hierarchy to make it to preset)... instead of displaying Neslihan's list, it would instead display the option "add as custom filter", which would then add the user's input to the list of types. We may or may not be able to get this to persist across app restarts (within the time allocated for this task), so there may be the inconvenience of having to create the custom filter again next time, but at least the option exists?

[neslihanturan]

@misaochan agreed about checkboxes, I will search for the best way of displaying checked options on search row. Because I wouldn't prefer to expand the check list and review what I clicked there. Instead I would prefer to see what I included when check list is collapsed. On the other hand, we can not fit 40 element to one row if all of them are checked.

@VojtechDostal thanks for pointing out "except" case, I will reconsider the design accordingly.

[neslihanturan]

Based on the discussions above, using checkbox seemed most appropriate method to select/deselect all and exclude some items.

My first suggestion is using checkbox with chips. When we checked some, they will be added to row as chips. The only concern of mine about this approach is, we may have 20 selected place types. Adding 20 chips to a row is open to discussion.

The second suggestion is using a text field, without displaying chips. User can only see the umber of checked place types with this option. To see their names, they should click edit text and they will be listed below. But I would prefer the first design.

[misaochan]

I like suggestion 2 a lot more. :) With suggestion 1, there may be far too many types to display if the user selects "all except X". Displaying selected types at the top is a nice touch, too, but I'm not sure about this because the list of selected types could, again, get extremely long, forcing the user to scroll down to see the actual type suggestions.

If the user searches for a place type that isn't in the list, there should be an option to "add custom filter".

The pin styling according to place states is a nice idea, but I'm not sure how it would work. Firstly, I think we only have two useful states: "past object" (which is basically 'doesn't exist', but I think it's clearer because it indicates that the object used to exist), and "has Wikipedia article". Also, AFAIK the two states are not necessarily mutually exclusive - a place could both be a past object AND have a Wikipedia article (@nicolas-raoul can you confirm this?). So in that case what color would we make it?

[neslihanturan]

If the user searches for a place type that isn't in the list, there should be an option to "add custom filter".

But I think @nicolas-raoul mentioned that this could take too much time. But if we still want to include this, I can add such think into design 2

[nicolas-raoul]

I am OK with both Nes' suggestions 1 and 2.

The performance problem is not only about custom filters, but might be also for built-in place types. For instance, let's say I select "school": I would probably expect the app to also include sub-classes of "school", such as "physico-mathematical boarding school" https://www.wikidata.org/wiki/Q4483543 . That is a rather time-consuming SPARQL request to perform.
I suggest running performance tests, here is how to get sub-classes: https://opendata.stackexchange.com/questions/9591/how-to-include-sub-classes-in-a-wikidata-sparql-query-example-when-querying
If it is too slow, a possible strategy could be to first get all items, and then send a request per item to get its details (ended or not, destroyed or not, all of its parent classes, whether it has a P18 or not, its label). More request, but the first results would surely come faster.

Personally I might not use Place types, because in my trips I am never overwhelmed by "too many items". I can understand that might be desirable by VojtechDostal if every single street of his city is an item, though.

Personally I would certainly use Place state, though. By the way, if I understand correctly "has photo" is the opposite of "needs photo", right? So can it not be simplified to 2 checkboxes?

[v] Exists [v] Needs photo -> Only shows current items (so, excluding those which were destroyed or ended) that do not have a P18 yet
[v] Exists [ ] Needs photo -> Shows all current items, with or without a P18
[ ] Exists [v] Needs photo -> Shows all items (including destroyed/ended ones) that need a photo
[ ] Exists [ ] Needs photo -> Show all items

I suggest the first (or maybe third) state to be the default one.

I agree with Josephine that pin colors are best used to show place states, rather than place types.
In a far future phase, pin shapes would be really cool, for instance a pin shaped as a train to indicate a train station, etc... but there are many many more important things to implement before ^_^

[neslihanturan]

[v] Exists [v] Needs photo -> Only shows current items (so, excluding those which were destroyed or ended) that do not have a P18 yet
[v] Exists [ ] Needs photo -> Shows all current items, with or without a P18

I agree that "exists" covers all items but on the other hand it can be confusing to see "needs photo" images when it is deselected. So the question is should we first filter as "need/not need photo" then filter as "exists", or first as "exists" then "need/not needed".

[ ] Exists [v] Needs photo -> Shows all items (including destroyed/ended ones) that need a photo

To make it more clear, we can rename it to "only exists".

[ ] Exists [ ] Needs photo -> Show all items

Maybe "Only needs photo" would be better for this one then. So that the opposite would be all.

Additionally,

I agree with Josephine that pin colors are best used to show place states, rather than place types.

It already does indicate place states. But even if we reduce number of place states to two, I think we should preserve pin styles as 3, to be more clear.

[misaochan]

• Re: place states, why would we need "Needs photo" or "has photo" at all? After all, Nearby by its very definition is "nearby places that need photos", so every single Wikidata item that we even fetch would need a photo and not have a photo. I would think that place types just needs (1) "still exists" and (2) "has Wikipedia article"?

• Re: place types, actually the implementation I had in mind was simple string matching, not a separate SPARQL query. We already fetch instanceOf as part of our initial Nearby query (that is, after all, how the "description" value of a place is poupulated), so once the users select a particular place type, we just need to go through all the items that we have fetched and remove the ones whose instanceOf field matches a deselected checkbox. All done internally, no additional queries. This would easily work with custom filters because a custom filter is just a string after all.

An additional SPARQL query for place type filters would be detrimental for the user I think, as they would need to wait for Nearby to load all over again whenever a filter is modified. However it does have the advantage of being able to handle subclasses. But it would be much more time consuming to implement.

[neslihanturan]

Re: place states, why would we need "Needs photo" or "has photo" at all? After all, Nearby by its very definition is "nearby places that need photos", so every single Wikidata item that we even fetch would need a photo and not have a photo. I would think that place types just needs (1) "still exists" and (2) "has Wikipedia article"?

I think we considered including Nearby Places Has Photo several times by saying it can help users to see example nearby photos. And including both would help us to solve the confusion of "what nearby photos means"

Re: place types, actually the implementation I had in mind was simple string matching, not a separate SPARQL query. We already fetch instanceOf as part of our initial Nearby query (that is, after all, how the "description" value of a place is poupulated), so once the users select a particular place type, we just need to go through all the items that we have fetched and remove the ones whose instanceOf field matches a deselected checkbox. All done internally, no additional queries. This would easily work with custom filters because a custom filter is just a string after all.

I agree to filtering what we already have

[nicolas-raoul]

Seeing existing pictures is very useful in all places that are not super-touristic.
Looking at it, if I see almost no pictures on the map, I know that I should shoot everything interesting, whereas if there are many pictures it means that probably everything has been shot already I move on quickly.
Especially in the countryside, most interesting places do not have a Wikidata item in the first place.

[misaochan]

Hmm, so in that case we are intending to merge viewing of existing pictures with Nearby? I was originally envisioning that we would eventually have a separate map of existing pictures in Explore, where it seems to be more appropriate in my opinion.

If you guys strongly feel that it should be merged with Nearby, I am OK with that, but @neslihanturan you do realize that this will increase your workload for this task hugely? Since we have not actually implemented a map of existing pictures, so you would need to handle all of that as well.

[neslihanturan]

When I consider our future new UI plan which will be implemented at the end of 2019 and will include bottom tabs as Home, Nearby, Explore, My contributions, I thought it can be confusing to have 2 maps on both Nearby(we consider renaming nearby though) and Explore tab. So my vote is handling all map related tasks from one place.

[misaochan]

@neslihanturan OK, I will leave the decision up to you. :) If you find that implementing a map of existing pictures takes too much time, you can just remove the filter for that and create an issue for a future enhancement. Maybe for next year's GSoC/Outreachy? ;)

[nicolas-raoul]

Not that I have any power to decide, but as a user I would probably find it more convenient to have a single map. The name "Nearby" can be used for both nearby images and nearby places that lack images.

[neslihanturan]

@nicolas-raoul I assumed Nearby naming will change not because has/lack images, it was because of search this area button. Sorry for mixing all issues together, but when it comes to UI decisions I try to connect dots.

To sum up the issue, it seems like we will go with the design 2, try to have a single map and have 2 chips instead of 3 (with better namings).

Update: We have one more additional place type as "Freedom of Panorama", see issue #2952 for more info.

[misaochan]

Sounds good!

Re: the naming, yeah, I think we were discussing this in another issue #2190 . I think the conclusion was that we want to stick to Nearby?