Hashes/encodings below the heuristic limit are treated as typos

[halkeye]

error: `Ba` should be `By`, `Be`
  --> ./content/blog/2017/08/2017-08-08-introducing-jenkins-minute.adoc:39:14
   |
39 | video::FhDomw6BaHU[youtube, width=852, height=480]

I can add it to [default.extend-identifiers] so its not a blocker but figured you'd like another test case.

[epage]

The challenge is being able to identify that has a hash. How do we tell a hash from an identifier?

Right now, we support

• SHA detection: must be 32+ characters long and consistent case
• Base 64 detection: Must be 90 characters long or have + / / in it and must have the padding bytes (though there is uncertainty if the padding byte requirement will stay, see Unexpectedly treated base64 value as a typo #413)

[halkeye]

yea that makes a lot of sense. I'm just starting to use the app and loving it so far, I've only had to whitelist two hashes that have ba in them so its not a big deal for me.

Maybe some sort of regex or something so I could whitelist video::[a-zA-Z0-9]\[

[pums974]

I think it causes also some problems with jupyter notebooks.

error: `ba` should be `by`, `be`
  --> jupyter.ipynb:661:11
    |
661 |    "id": "6ba7c279",
    |            ^^
    |
error: `ba` should be `by`, `be`
  --> jupyter.ipynb:784:15
    |
784 |    "id": "33088ba8",
    |                ^^
    |
error: `ba` should be `by`, `be`
  --> jupyter.ipynb:1029:10
     |
1029 |    "id": "ba6788ca",
     |           ^^
     |
error: `ba` should be `by`, `be`
  --> jupyter.ipynb:2029:10
     |
2029 |    "id": "ba638183",
     |           ^^
     |

[tspearconquest]

Hello,

Git commit hashes tend to run in the range [0-9a-fA-F]{7,} so that would be a useful addition.

[epage]

@tspearconquest for shorter git commit hashes, we'll need to rely on a heuristic like talked about in #484 because shorter commit hashes could just as easily be words.

[jplatte]

How about adding a heuristic "word contains characters preceded by numbers" (where "word" is a whitespace-separated segment, not a case-separated segment)? I don't think I've ever seen an identitifer be named foo1bar or 3foo, though foo3 or foo3_bar seem realistic (e.g. zip3, zip4).

[halkeye]

sha1hash?

[jplatte]

Right, there's a few exceptions (I also remembered there being 2to3), but maybe it's still a good heuristic? Personally, I consider false positives a bigger issue than false negatives, and I think that matches typos' overall approach.

[pums974]

There is also all the this2that and thing4stuff

[epage]

@jplatte I'd probably refine your comment to be "any identifier that exclusively word splits due to numbers and not any other separator (be it case or _)

The next question is the likelihood of a shortened sha having no numbers. I probably didn't bring this up in the other thread talking about heuristics but I suspect to have something always complain than it have it complain in a way people no longer expect.