code review fixes

docs/API.md

@@ -340,7 +340,7 @@ Topic `fnd` is automatically created for every user at the account creation time
 
 A tag is an arbitrary case-insensitive Unicode string (forced to lowercase on the server) up to 96 characters long which may contain characters from `Letter` and `Number` Unicode [classes/categories](https://en.wikipedia.org/wiki/Unicode_character_property#General_Category) as well as any of the following ASCII characters: `_`, `.`, `+`, `-`, `@`, `#`, `!`, `?`.
 
-Tag may have a prefix which serves as a namespace. The prefix is a 2-16 character string which starts with a letter [a-z] and may contain ASCII letters and numbers followed by a colon `:`, ex. prefixed phone tag `tel:+14155551212` or prefixed email tag `email:alice@example.com`. Some prefixed tags are optionally enforced to be unique. In that case only one user or topic may have such a tag. Certain tags may be forced to be immutable to the user, i.e. user's attempts to add or remove an immutable tag will be rejected by the server.
+Tag may have a prefix which serves as a namespace. The prefix is a 2-16 character string which starts with a letter [a-z] and may contain lowercase ASCII letters and numbers followed by a colon `:`, ex. prefixed phone tag `tel:+14155551212` or prefixed email tag `email:alice@example.com`. Some prefixed tags are optionally enforced to be unique. In that case only one user or topic may have such a tag. Certain tags may be forced to be immutable to the user, i.e. user's attempts to add or remove an immutable tag will be rejected by the server.
 
 The tags are indexed server-side and used in user and topic discovery. Search returns users and topics sorted by the number of matched tags in descending order.
 
@@ -358,7 +358,7 @@ _CURRENTLY UNSUPPORTED_ When a new user registers with tags matching the given q
 
 Tinode query language is used to define search queries for finding users and topics. The query is a string containing atomic terms separated by spaces or commas. The individual query terms are matched against user's or topic's tags. The individual terms may be written in an RTL language but the query as a whole is parsed left to right. Spaces are treated as the `AND` operator, commas (as well as commas preceded and/or followed by a space) as the `OR` operator. The order of operators is ignored: all `AND` tags are grouped together, all `OR` tags are grouped together. `OR` takes precedence over `AND`: if a tag is preceded of followed by a comma, it's an `OR` tag, otherwise an `AND`. For example, `aaa bbb, ccc` (`aaa AND bbb OR ccc`) is interpreted as `(bbb OR ccc) AND aaa`.
 
-Query terms containing spaces must convert spaces to underscores ` ` -> `_`.
+Query terms containing spaces must convert spaces to underscores ` ` -> `_`, e.g. `new york` -> `new_york`.
 
 **Some examples:**
 * `flowers`: find topics or users which contain tag `flowers`.
@@ -375,7 +375,13 @@ The incremental update request is processed left to right. It may contain the sa
 
 #### Query Rewrite
 
-Finding users by login, phone or email requires query terms to be written with prefixes, i.e. `email:alice@example.com` instead of `alice@example.com`. This may present a problem to end users because it requires them to learn the query language. Tinode solves this problem by implementing _query rewrite_ on the server: if query term (tag) does not contain a prefix, server rewrites it by adding the appropriate prefix. In queries to `fn.public` the original term is also kept (query `alice@example.com` is rewritten as `email:alice@example.com OR alice@example.com`), in queries to `fn.private` only the rewritten term is kept (`alice@example.com` is rewritten as `email:alice@example.com`). All terms that look like email, for instance, `alice@example.com` are rewritten to `email:alice@example.com OR alice@example.com`. Terms which look like phone numbers are converted to [E.164](https://en.wikipedia.org/wiki/E.164) and also rewritten as `tel:+14155551212 OR +14155551212`. In addition, in queries to `fn.public` all other unprefixed terms which look like logins are rewritten as logins: `alice` -> `basic:alice OR alice`.
+Finding users by login, phone or email requires query terms to be written with prefixes, i.e. `email:alice@example.com` instead of `alice@example.com`. This may present a problem to end users because it requires them to learn the query language. Tinode solves this problem by implementing _query rewrite_ on the server: if query term (tag) does not contain a prefix, server rewrites it by adding the appropriate prefix. In queries to `fnd.public` the original term is also kept (query `alice@example.com` is rewritten as `email:alice@example.com OR alice@example.com`), in queries to `fnd.private` only the rewritten term is kept (`alice@example.com` is rewritten as `email:alice@example.com`). All terms that look like email, for instance, `alice@example.com` are rewritten to `email:alice@example.com OR alice@example.com`. Terms which look like phone numbers are converted to [E.164](https://en.wikipedia.org/wiki/E.164) and also rewritten as `tel:+14155551212 OR +14155551212`. In addition, in queries to `fnd.public` all other unprefixed terms which look like logins are rewritten as logins: `alice` -> `basic:alice OR alice`.
+
+As described above, tags which look like phone numbers are converted to E.164 format. Such conversion requires a country code. The following logic is used when converting phone number tags to E.164:
+* If the tag already contains a country code prefix, it's used as is: `+1(415)555-1212` -> `+14155551212`.
+* If the tag has no prefix, country code is taken from the locale value set by the client in `lang` field of the `{hi}` message.
+* If client has not provided the code, the country code is taken from `default_country_code` field of the `tinode.conf`.
+* If no `default_country_code` is set in `tinode.conf`, `US` country code is used.
 
 #### Possible Use Cases
 * Restricting users to organisations.

server/utils.go

@@ -418,10 +418,7 @@ func filterRestrictedTags(tags []string, namespaces map[string]bool) []string {
 // rewriteToken attempts to match the original token against the email, telephone number and optionally login patterns.
 // The tag is expected to be converted to lowercase.
 // On success, it prepends the token with the corresponding prefix. It returns an empty string if the tag is invalid.
-// TODO: better handling of countryCode:
-// 1. As provided by the client (e.g. as specified or inferred from client's phone number or location).
-// 2. Use value from the .conf file.
-// 3. Fallback to US as a last resort.
+// TODO: consider inferring country code from user location.
 func rewriteTag(orig, countryCode string, withLogin bool) string {
 	// Check if the tag already has a prefix e.g. basic:alice.
 	if prefixedTagRegexp.MatchString(orig) {
@@ -433,6 +430,7 @@ func rewriteTag(orig, countryCode string, withLogin bool) string {
 		if len([]rune(addr.Address)) < maxTagLength && addr.Address == orig {
 			return "email:" + orig
 		}
+		log.Println("failed to rewrite email tag", orig)
 		return ""
 	}
 
@@ -450,6 +448,9 @@ func rewriteTag(orig, countryCode string, withLogin bool) string {
 	if tagRegexp.MatchString(orig) {
 		return orig
 	}
+
+	log.Printf("invalid generic tag '%s'", orig)
+
 	return ""
 }