Nearby for super-popular item: stays grey until "Could not load place data Unexpected response code"

[nicolas-raoul]

Maybe the Eiffel Tower item has more data than we expect, or something different that we do not support yet?

[savsch]

The corresponding query times out:
Response for this request:

HTTP/2 500 Internal Server Error
Server: nginx/1.18.0
Date: Mon, 23 Dec 2024 22:34:00 GMT
Content-Type: text/plain;charset=utf-8
X-First-Solution-Millis: 12
X-Served-By: wdqs2011
Access-Control-Allow-Origin: *
Vary: Accept-Encoding
Age: 62
X-Cache: cp5021 miss, cp5021 pass
X-Cache-Status: pass
Server-Timing: cache;desc="pass", host;desc="cp5021"
Strict-Transport-Security: max-age=106384710; includeSubDomains; preload
Report-To: { "group": "wm_nel", "max_age": 604800, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
Nel: { "report_to": "wm_nel", "max_age": 604800, "failure_fraction": 0.05, "success_fraction": 0.0}
X-Client-Ip: ✂✂SNIP✂✂

SPARQL-QUERY: queryStr=SELECT
  ?item
  (SAMPLE(?label) AS ?label)
  (SAMPLE(?class) AS ?class)
  (SAMPLE(?description) AS ?description)
  (SAMPLE(?classLabel) AS ?classLabel)
  (SAMPLE(?pic) AS ?pic)
  (SAMPLE(?destroyed) AS ?destroyed)
  (SAMPLE(?endTime) AS ?endTime)
  (SAMPLE(?wikipediaArticle) AS ?wikipediaArticle)
  (SAMPLE(?commonsArticle) AS ?commonsArticle)
  (SAMPLE(?commonsCategory) AS ?commonsCategory)
WHERE {
  SERVICE <https://query.wikidata.org/sparql> {
    values ?item {
      
wd:Q243
wd:Q162873
    }
  }

  # Get the label in the preferred language of the user, or any other language if no label is available in that language.
  OPTIONAL {?item rdfs:label ?itemLabelPreferredLanguage. FILTER (lang(?itemLabelPreferredLanguage) = "en")}
  OPTIONAL {?item rdfs:label ?itemLabelAnyLanguage}
  BIND(COALESCE(?itemLabelPreferredLanguage, ?itemLabelAnyLanguage, "?") as ?label)

  # Get the description in the preferred language of the user, or any other language if no description is available in that language.
  OPTIONAL {?item schema:description ?itemDescriptionPreferredLanguage. FILTER (lang(?itemDescriptionPreferredLanguage) = "en")}
  OPTIONAL {?item schema:description ?itemDescriptionAnyLanguage}
  BIND(COALESCE(?itemDescriptionPreferredLanguage, ?itemDescriptionAnyLanguage, "?") as ?description)

  # Get the class label in the preferred language of the user, or any other language if no label is available in that language.
  OPTIONAL {
  ?item p:P31/ps:P31 ?class.
    OPTIONAL {?class rdfs:label ?classLabelPreferredLanguage. FILTER (lang(?classLabelPreferredLanguage) = "en")}
    OPTIONAL {?class rdfs:label ?classLabelAnyLanguage}
    BIND(COALESCE(?classLabelPreferredLanguage, ?classLabelAnyLanguage, "?") as ?classLabel)
  }

  OPTIONAL {
  ?item p:P31/ps:P31 ?class.
  }

  # Get picture
  OPTIONAL {?item wdt:P18 ?pic}

  # Get existence
  OPTIONAL {?item wdt:P576 ?destroyed}
  OPTIONAL {?item wdt:P582 ?endTime}

  # Get Commons category
  OPTIONAL {?item wdt:P373 ?commonsCategory}

  # Get Wikipedia article
  OPTIONAL {
    ?wikipediaArticle schema:about ?item.
    ?wikipediaArticle schema:isPartOf <https://en.wikipedia.org/>. # TODO internationalization
  }

  # Get Commons article
  OPTIONAL {
    ?commonsArticle schema:about ?item.
    ?commonsArticle schema:isPartOf <https://commons.wikimedia.org/>.
  }
}
GROUP BY ?item

java.util.concurrent.TimeoutException
	at java.util.concurrent.FutureTask.get(FutureTask.java:205)
	at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:292)
	at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:678)
	at com.bigdata.rdf.sail.webapp.QueryServlet.doGet(QueryServlet.java:290)
	at com.bigdata.rdf.sail.webapp.RESTServlet.doGet(RESTServlet.java:240)
	at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doGet(MultiTenancyServlet.java:273)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
	at org.wikidata.query.rdf.blazegraph.throttling.ThrottlingFilter.doFilter(ThrottlingFilter.java:322)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.wikidata.query.rdf.blazegraph.throttling.SystemOverloadFilter.doFilter(SystemOverloadFilter.java:84)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at ch.qos.logback.classic.helpers.MDCInsertingServletFilter.doFilter(MDCInsertingServletFilter.java:50)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.wikidata.query.rdf.blazegraph.filters.QueryEventSenderFilter.doFilter(QueryEventSenderFilter.java:125)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.wikidata.query.rdf.blazegraph.filters.ClientIPFilter.doFilter(ClientIPFilter.java:43)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.wikidata.query.rdf.blazegraph.filters.JWTIdentityFilter.doFilter(JWTIdentityFilter.java:66)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.wikidata.query.rdf.blazegraph.filters.RealAgentFilter.doFilter(RealAgentFilter.java:33)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.wikidata.query.rdf.blazegraph.filters.RequestConcurrencyFilter.doFilter(RequestConcurrencyFilter.java:50)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:503)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
	at java.lang.Thread.run(Thread.java:750)

[savsch]

It turns out that this clause is causing the problem:

OPTIONAL {?item schema:description ?itemDescriptionPreferredLanguage. FILTER (lang(?itemDescriptionPreferredLanguage) = "en")}

Removing it makes the query succeed.

[nicolas-raoul]

Thanks a lot for investigating! I guess it happens because this item (and similarly popular items) have a description in many languages.
I wonder what we should do. Maybe if we get timeout then we can try again with several smaller queries (for instance one to get descriptions and one to get the rest), or maybe even load no description.

The goal of this app is to get people to upload pictures of underrepresented things, and most items that need a picture do not have this issue, so it is not really high priority compared to other Nearby issues. :-)

[sonalyadav1]

I'll work on this issue.

[nicolas-raoul]

Random thought, how about the following:

1. Catch the timeout exception.
2. If the timeout exception happened only for a minority of pins, silently mark the timeout pin(s) as green, instead of showing an error snack and leaving the pin grey.

Reasoning:

• If we get a timeout, it probably means that this is a super-popular item, and thus it most probably has a P18 already.
• But if all (most?) pins got timeout then it might be a server issue rather than the issue being super-popular.

[sonalyadav1]

Interesting idea! Makes sense — handling isolated timeouts silently could improve UX, especially if the item is likely to have P18. And yeah, if it affects most pins, that does point more to a server-side issue. Worth exploring!