Flash-write repetitively missing bytes on 0x100-boundary

[mafiltenborg]

Writing Hex-file to ATmega644PA using below chip_dict, a Feather nRF52840 and a bit of wiring to the ISP target. Works beautifully and validates - but target code does not run as expected.

.
.
.
# Processor definition
atmega644pa = {'name': "ATmega644PA"}
atmega644pa['sig'] = [0x1E, 0x96, 0x0A]
atmega644pa['flash_size'] = 65536
atmega644pa['page_size'] = 256
atmega644pa['fuse_mask'] = (0xF7, 0x8F, 0xFD, 0xFF) # Low, High, Extended, Lockbits
.
.
print("Programming flash with",filename)
if not avrprog.program_file(atmega644pa, filename, verbose=False, verify=False):
    error("Failure programming flash with" + filename )
else:
    print("Programmed OK")
time.sleep(1)

print("Verifying flash against",filename)
if not avrprog.verify_file(atmega644pa,filename,verbose=False):
    error("Failure verifying flash against " + filename )
else:
    print("Validated OK")

creates this screen output

Working with an ATmega644PA CPU. Good!
Read lockbits: ['0xf7', '0x8f', '0xfd', '0xff']
Erasing chip...
Re-read lockbits: ['0xf7', '0x8f', '0xfd', '0xff']
Programming flash with SENSORCARD644_V7.hex
Programmed OK
Verifying flash against SENSORCARD644_V7.hex
Validated OK
Lockbits now:  ['0xf7', '0x8f', '0xfd', '0xff']
Writing fuses & lockbits
Re-read lockbits: ['0xf7', '0x8f', '0xfd', '0xff']
Programming done!

Looks good. However, target doesn't function. So I read the flash content back again using AtmelStudio+AVRISPmk2 and this shows something is not right.

Every 0x100 bytes a block of 0xFFs appear in the read-back content, effectively being unprogrammed 'holes' in the binary.

What triggers my suspicion is that these holes all start on ..00, and their size goes from tiny to small, proceeding over medium to largish, then big - in a linear fashion. And then back to tiny to repeat the cycle.
Hole size 0 to 18 bytes.

Some kind of overrun in a counter?

I've attached a screenshot of a binary diff, exposing the phenomenon; readback on left, original on right.

AVRprog taken from Circuitpython bundle 20211024

[ladyada]

could be! we've never programmed that chip, if you find and fix the bug please submit a PR :)

[mafiltenborg]

The thing is; I'm not proficient with neither Python nor Github. Not sure you'd want me to submit a correction; it'd probably do more damage than good. What I can offer to the library maintainer with confidence is test-data down to bare metal showing actual finds.

There IS a library maintainer?

Will procede to test with one of the 'known' chips; a 2560 which we also happen to use in our products. If the error persists, I'll provide more info.

could be! we've never programmed that chip, if you find and fix the bug please submit a PR :)

[ladyada]

ok!

[mafiltenborg]

Okay. I've done some experiments, and found that hex-files holding 32 bytes per line are conveyed properly. However, the same binary content formatted in lines holding 20 bytes each, fail on 256-byte boundaries - the pagesize number. An additional 4 bytes get written each time. 13 lines each holding 20 bytes = 260 bytes. Pagesize 256. Delta; 4 bytes.
I'm guessing the read_hex_page() routine does not like working with files that don't use line-lengths not lining up with pagesize. Leaving this info for someone who's better at working with github as well as python. Juicy zip with testcode, evidence and a readme attached.
AVR_feather.zip

[will127534]

I've also encounter this problem, and here is the issue and my fix:

Here is the HEX code that causes the problem:
:10 2B8C 00 010000D84300010001050004040D0E13 E0
:10 2B9C 00 1516191A1C232526292A2C3234000000 5C
:10 2BAC 00 00 E30B5F0BAC0B8A0B9E0B2A0C000000 96
:10 2BBC 00 001F011B011701F50A03012E012A0126 32
:10 2BCC 00 0100000000C501B101AA01A701A401A3 E5
:10 2BDC 00 0100000000B008940624027F07010160 88
:10 2BEC 00 0158017803A90352014F014C01490142 DC
:10 2BFC 00 0155013F X 013C0139013601A7060D0A00 C0
*********************** ^ Where the page boundary is in the line
:10 2C0C 00756E53796E63004552524F52002C006E 14
:0C 2C1C 00616E00696E66006F76660000 55

And here is the page buffer shown by the code:
Programming page $2C00...From HEX file:
['0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0x75', '0x6e', '0x53', '0x79', '0x6e', '0x63', '0x0', '0x45', '0x52', '0x52', '0x4f', '0x52', '0x0', '0x2c', '0x0', '0x6e', '0x61', '0x6e', '0x0', '0x69', '0x6e', '0x66', '0x0', '0x6f', '0x76', '0x66', '0x0', '0x0', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff', '0xff']

You can see that it is missing the data after the breaks, which is the root cause.
Apparently the code doesn't consider that case, so my brutal fix is to move one line back every successful read at #L486