Filesystem fails to write file when there should be enough space available

[microbit-carlos]

Edit: You can ignore the contents of this first message and start reading from comment #105 (comment)

---

Flashing this large Python file that tries to constantly write a tello.cfg file in the filesystem, which is less than a single fs block, works correctly for the first 19 writes, but then it fails:
fs-error.py

...
18
1 44000
2 4400
3 44000
4 43984
5 3712
6 43712
19
1 43712
2 43712
3 43712
[Errno 28] ENOSPC
MicroPython 569e361 on 2022-06-20; micro:bit v2.0.0 with nRF52833
Type "help()" for more information.

As the file is less than a fs block in size, and is constantly overwriting it with the same file name, any used fs blocks should be free to be reused, so there should always be available space.

The 1 xxxx, 2 xxxx, 3 xxxx lines are printing the gc.mem_free() output, so there is plenty of free RAM to hold a full page of flash into RAM.

Something might be going wrong at the point the file system is sweeping? https://github.com/bbcmicrobit/micropython/blob/v1.0.1/source/microbit/filesystem.c#L151

I cannot reproduce it with this smaller similar example:

from microbit import *

id = 0

def write_configuration(ssid,tx,rx,ID):
    global id
    id = ID
    cfg=open('tello.cfg','w')
    cfg.write(ssid + "@" + tx + "$" + rx + "%" + id)
    cfg.close()
    return("ok")
    
i = 0
while True:
    write_configuration("TELLO-9F856A", "pin2", "pin1", "9")
    i += 1
    print("Attempt {}: ".format(i))

[microbit-carlos]

Okay, so removing code function by functions it gets to a point where the fs doesn't error anymore.

So, I've created this simplified version, in this case it has around 15KBs of comments at begging of the file and then this test code writing a file:
fs_error_simpler.py.zip

# 12345678901234567

import gc

i = 0
while True:
    i += 1
    print("Attempt {}".format(i))
    print(1, gc.mem_free())
    f = open('file.txt','w')
    print(2, gc.mem_free())
    f.write("something")
    f.close()

And it fails after 32 writes:

...
Attempt 32
1 61472
Traceback (most recent call last):
  File "main.py", line 169, in <module>
OSEror: [Errno 28] ENOSPC
MicroPython 569e361 on 2022-06-20; micro:bit v.0.0 with nRF52833
Type "help()" for more information.
>>> 

Removing a single character from the header (so making main.py a single byte shorter, to 16118 bytes) fixes the issue.

So, those numbers make this a lot more obvious. We don't have a full flash page of "freed" blocks that can be erased.

Basically we have 24 KBs for the filesystem, 1 page (4 KBs) is the scratch page, leaving 20 KBs for files. This error appears when main.py is larger than 16 KBs (in physical flash space, not file size).

So, with this example script, we have 31 free fs-blocks (128 bytes each), once they are all written into, even if they are all freed, the flash page that contains all these freed blocks also has a block used by main.py

@dpgeorge I assumed at this point the microbit-fs would call filesystem_sweep(), which would move all the used pages down one page (as by default the Python Editor places the scratch page at the end), and that would free up 31 blocks again?
https://github.com/bbcmicrobit/micropython/blob/v1.0.1/source/microbit/filesystem.c#L151

[microbit-carlos]

In case it's useful, this is a flash read back after running the example script in my last message.
Here we can see in address 0x71000 that main.py takes a block over 16 KBs, and all the blocks after that with filename file.txt have been freed until we reach the scratch page at address 0x72000:
flash-hex.txt

And for info as well, the fs location and size are defined in:
https://github.com/microbit-foundation/micropython-microbit-v2/blob/v2.0.0/src/codal_port/filesystem.ld

[dpgeorge]

Fixed by b781e65

The problem was that the filesystem would not sweep and reclaim the usable chunks if the number of free chunks was less than 32. With the fix it now sweeps if there are any free chunks, so all available space can now be used.

[dpgeorge]

@microbit-carlos should we apply this patch also to micro:bit v1?

[microbit-carlos]

Thanks Damien. And yeah, taking in consideration the fix shouldn't change the memory footprint, we should add it to V1 as well 👍

[dpgeorge]

OK, same patch is now applied to micro:bit v1.

[martinwork]

From support ticket https://support.microbit.org/helpdesk/tickets/56425 (private)

When will this fix be added to the micropython build that is used in the online python editor?

[microbit-carlos]

Yes, we'll make a release very soon, which will include this fix.