Remove case insensitivity by changing the approach towards style insensitivity

[IgorGRBR]

Abstract

Instead of removing underscores and forcing every letter to lowercase, replace every occurrence of underscore followed by any-case letter with uppercase letter in snake_case identifiers, and keep camelCase identifiers as-is.

Motivation

Current style insensitivity algorithm can produce a few problems internally when trying to represent compound words that can be split more than one way in identifier names, as the internal all-lowercase identifier names are indistinguishable in certain scenarios (gameStop vs gamesTop, index vs inDEX, superb_owl vs super_bowl, island vs isLand, etc).

Description

I propose for identifiers to store their names with camelCase (or snake_case, or any other way to store separate words inside of a identifier), as a case of a letter or an underscore in the name clearly delimits the words used inside that name. This should resolve the issue presented above by allowing separate words to be represented properly.

Conversion between styles would preserve the words by forcing any letter preceded by an underscore to be uppercase when converting from snake_case to camelCase. When converting from camelCase to snake_case, just reverse the process.

Code Examples

No response

Backwards Compatibility

Macros will break if they generate an all-lowercase identifiers, but other than that I'm not aware of any other backwards-incompatible changes this would introduce to Nim.

[ASVIEST]

As for me better to add (again) style skins (but more generalized):

# possible syntax
import strutils

proc processIdent(s: string): string =
  var afterWhitespace = false
  for i in 0..<s.len - 1:
    if s[i] == '_' and s[i + 1] != '_': result.add s[i + 1].toUpperAscii
    elif not afterWhitespace: result.add s[i]
    afterWhitespace = s[i] == '_' and s[i + 1] != '_'
  if not afterWhitespace: result.add s[^1]

proc sameIdent(a, b: string): bool =
  processIdent(a) == processIdent(b)

import std/syntaxskins # it contains also default skins
const mySkin = initSyntaxSkin(sameIdent = sameIdent)
{.syntaxSkin: mySkin.}

It also solves problem with C wrappers (you can just disable case insensitivity for wrapper module and use case sensitivity wrapper from other module in case sensitivity manner)
And also you still have the benefits of case insensitivity (in other modules)
but it realy hard to implement especially executing pragmas during parsing (for style skins in general)
ideally the compiler should also have support for jit compilation for vm

about interaction between modules:

# gl.nim
import std/syntaxskins
{.syntaxSkin: caseSensive.} # or wrapperDefault, etc...
const GL_FLOAT = 0x1406
type GLfloat* = float32

# code.nim
import gl
# we have default syntax skin
let x = GL_FLOAT # external idents used with it's sameIdent logic
proc test(x: GLfloat) = discard
# but non external identifiers have normal logic
import snake_case_module
var x = weirdName # in snake_case_module it is weird_name

[IgorGRBR]

Style skins seems like an interesting idea, but a bit irrelevant here as this RFC tries to address the issue of having multiple identifiers with different compound words that are treated as the same identifier, because the algorithm that converts them to an IR can't handle such cases.

If your snake_case_module contains both weird_name and weirdname, Nim will still treat it as the same. Sure, you could just add {.syntaxSkin: caseSensive.} into snake_case_module, but I think it would be better to make Nim behave properly by default.

[metagn]

Macros will break if they generate an all-lowercase identifiers, but other than that I'm not aware of any other backwards-incompatible changes this would introduce to Nim.

Isn't the suggestion to remove case insensitivity? That is definitely breaking to code which uses different casings.

[ASVIEST]

Perhaps you still need to add a rule:
letters between two capital letters must be capitalized
GlFloat -> GLFloat
this is needed for something like this
Gl_float -> GlFloat -> GLFloat

It solves @metagn cases
newHtmlParser (with originaly newHtmlParser) -> newHTMLParser
new_html_parser -> newHtmlParser -> newHTMLParser
so new_html_parser match newHTMLParser

[ASVIEST]

Macros will break if they generate an all-lowercase identifiers, but other than that I'm not aware of any other backwards-incompatible changes this would introduce to Nim.

Isn't the suggestion to remove case insensitivity? That is definitely breaking to code which uses different casings.

It is my understanding that it about changing the register insensitivity rules, not removing it.

[metagn]

Sorry, I was wrong about newHTMLParser, the suggestion is to transform from snake case to case sensitive camel case, we already accept that newHTMLParser and newHtmlParser are different in camel case so they should also be different in snake case.

[ASVIEST]

Sorry, I was wrong about newHTMLParser, the suggestion is to transform from snake case to case sensitive camel case, we already accept that newHTMLParser and newHtmlParser are different in camel case so they should also be different in snake case.

newHTMLParser and newHtmlParser are the same
(when stylecheck off)

[metagn]

With this proposal, they no longer become the same.

[ASVIEST]

Perhaps you still need to add a rule: letters between two capital letters must be capitalized GlFloat -> GLFloat this is needed for something like this Gl_float -> GlFloat -> GLFloat

It solves @metagn cases newHtmlParser (with originaly newHtmlParser) -> newHTMLParser new_html_parser -> newHtmlParser -> newHTMLParser so new_html_parser match newHTMLParser

this rule lets it be the same
and should maintain differences between proposed cases.

[ASVIEST]

import std/syntaxskins
{.syntaxSkin: caseSensive.} # or wrapperDefault, etc...

BTW, now I think that sameIdent shouldn't be in syntaxSkin, it should be another pragma like {.cmpIdent: sth.} (because it's not just about one module, it's about all it's private/public idents).
And this is easier to implement than it seems, because the idents are compared in sem via getIdent (https://github.com/nim-lang/Nim/blob/devel/compiler/idents.nim)

[IgorGRBR]

I think it would be better to convert every uppercase letter in a continuous sequence of uppercase letters to lowercase, except the first and the last ones, since a single identifier can have many words and (for me, at least) its not clear if compiler should alternate between upper and lower cases or make everything between first and last uppercase letter uppercase and what effects that may cause.

With this logic newHTMLParser becomes newHtmlParser, GLFloat -> GlFloat, Gl_float, etc.

What do you think?

[ASVIEST]

Maybe it's better, but I don't see much difference.
It's worth noting that this resolves this case anyway:

getISLandFrame -> getIsLandFrame
get_is_land_frame -> getIsLandFrame # match but shouldn't
# must be valid only:
get_isl_and_frame -> getIslAndFrame # no match but should

NOTE: for uppercase variant result is same
this can happen, although not often.

# l - lowercase letter
# u - uppercase letter
# (l or u)* - zero and more matches
# (l or u)+ - one and more matches

# string with pattern:
l*   u+  l+  u+  l* # in any case it will be the same

Idealy, and must be camel case (And), but we can't detect it

For getISLAndFrame it working as expected

[Araq]

I thought about making Nim case sensitive but allowing for a .insensitive pragma that applies for imported symbols. But it looks unreasonably hard to implement with unknown effects for template instantiations etc.

[IgorGRBR]

getISLandFrame -> getIsLandFrame
get_is_land_frame -> getIsLandFrame # match but shouldn't
# must be valid only:
get_isl_and_frame -> getIslAndFrame # no match but should

NOTE: for uppercase variant result is same this can happen, although not often.

I'm not sure I understand your examples correctly. get_is_land_frame and getIsLandFrame match and should match (both in current state of Nim and with proposed changes from this RFC). I don't see why get_isl_and_frame and getIslAndFrame woudn't match either (after both RFC changes and lowercasing letters between 2 capitals).

For getISLAndFrame it working as expected

Get getISLAndFrame is problematic if you want to convert letters between 2 capitals to uppercase, because both it and getIslandFrame will be converted to getISLANDFrame.