Add LFS_STATICCFG and other config improvements

[geky]

This is a big set of config changes that originally came from the idea in this comment: #158 (comment). This would be an API-breaking change (but not disk-breaking), so at the moment I am just looking for feedback.

The big change is the addition of static config, in which defining the relevant defines (LFS_READ_SIZE for example) will override the dynamic config found in lfs_cfg (lfs->cfg->read_size for example). If you only have a single filesystem configuration in your build, this lets the compiler optimize out logic depending on this configuration.

• Added static configs which allow optimizing out the lfs_cfg and lfs_file_cfg structs 5f885f0

• Change the *_max configurations to *_limit, since they are slightly different than the *_MAX macros 5f885f0

• Rename lfs_(mount|format|migrate) -> lfs_(mount|format|migrate)cfg if cfg is provided c44427f

  This matches the existing lfs_file_opencfg.

In addition, this PR contains a number of changes/tweaks/improvements to the config API of littlefs:

• Renamed the LFS_CONFIG macro to LFS_UTIL to avoid confusion with the lfs_cfg struct ef9ba2d

• Removed lingering clib includes in lfs.h ef9ba2d

• Moved error code definitions to lfs_util.h ef9ba2d

• Used proper stdint definitions in lfs_scmp ef9ba2d

• Rename *_config structs to *_cfg for consistency a549413

• Changed callbacks to take user provided context directly a7cdd56

    struct lfs_cfg {
        void *ctx;
  -     int (*read)(const struct lfs_cfg *cfg, ...);
  +     int (*read)(void *ctx, ...);
        ...

• Changed block device callbacks to use a bd_* prefix 190eb83

    struct lfs_cfg {
  -     void *ctx;
  -     int (*read)(void *ctx, ...);
  +     void *bd_ctx;
  +     int (*bd_read)(void *ctx, ...);
        ...

• Renamed cache_size -> buffer_size to avoid confusion around the usefulness of >block_size buffer sizes 4990837

I also explored copying the dynamic configuration into the lfs_t struct at runtime. This would have avoided needing to persist the lfs_cfg struct for the lifetime of littlefs. This exists on this branch here:
https://github.com/littlefs-project/littlefs/tree/config-improvements-mut

But I ended up dropping this change as it increased the code size and made the internals quite a bit more complicated in order to support both dynamic and static configurations. More info here: f9324d1. Though I am still open to bringing this in if it's believed to be a better user experience.

Open to thoughts or suggestions.

Supersedes #262

---

EDITed based on feedback, previously:

The big change is the addition of LFS_STATICCFG, a define which optimizes out the lfs_cfg struct if defined, allowing for smaller builds.

• Add LFS_STATICCFG to optimize out the lfs_cfg struct c44427f
• Add LFS_FILE_STATICCFG to optimize out the lfs_file_cfg struct c44427f
• Rename lfs_(mount|format|migrate) -> lfs_(mount|format|migrate)cfg if cfg is provided c44427f

This matches the existing lfs_file_opencfg.

[dpgeorge]

Reducing code size and providing static configuration, in general that sounds good to me!

[tim-nordell-nimbelink]

Food for thought, but another pattern could have been something like:

struct lfs_config
{
...
#ifndef LFS_STATICCFG_READ_SIZE
    lfs_size_t read_size;
#endif
#ifndef LFS_STATICCFG_WRITE_SIZE
...
};
...
static inline lfs_size_t lfs_cfg_read_size(const struct lfs_config *c)
{
#ifndef LFS_STATICCFG_READ_SIZE
    return c->read_size;
#else
    return LFS_STATICCFG_READ_SIZE;
#endif
}

That would allow for:

• Partially optimized builds for any configuration. E.g. a user could declare what variables they'd never change for a given static configuration (but maybe would change a different parameter), and this could account for that.
• You could add min/max checks for the dynamic and/or static configurations that are used across the board. For instance, the block cycles need to be either off, or at least 2 IIRC.
• Having the functions in the header (not strictly required) allows a single point of entry for users to read a given configuration item regardless of if it's static or dynamic. Right now all of the helpers are placed into the lfs.c file, but it'd be potentially useful for a single point of retrieval.

[tim-nordell-nimbelink]

(Granted, doing the above, would require a cfg object to be always passed around to all functions instead of being able to remove it as in this patch; but this could get optimized out with link time optimization.)

[geky]

Hmm, I'll have to see what this looks like. Partial optimizations would be nice, though it may be a bit difficult to track down all constant-related warnings.

Granted, doing the above, would require a cfg object to be always passed around to all functions

We could always do a big:

#ifdef LFS_STATICCFG_READ_SIZE && \
        LFS_STATICCFG_PROG_SIZE && \
        LFS_STATICCFG_BLOCK_SIZE && \
        LFS_STATICCFG_BLOCK_COUNT && \
        ...

You could add min/max checks for the dynamic and/or static configurations that are used across the board. For instance, the block cycles need to be either off, or at least 2 IIRC.

Currently these are checked by runtime asserts in lfs_init. Unfortunately the math isn't always decidable at compile-time by C, but the runtime checks can be removed if you define LFS_NO_ASSERTS.

[tim-nordell]

I went ahead and created something since I was curious. See here: tim-nordell@734de59

[tim-nordell]

Interestingly, if the avoidance of division in the eviction logic is applied (#489) in conjunction with a static configuration using powers of 2 for all parameters, we end up with only 1 function performing division with a static configuration: lfs_ctz_index(...)

That function does a non div-by-power-of-2 since it divides by the block size less 2 * 4. It looks like that function calculates something integral to the design of LFS, so the - (2 * 4) can't be removed (to optimize for architectures without a hardware divider).

[geky]

Oh! That is an interesting observation.

It doesn't surprise me that lfs_ctz_index gets in the way, it is the most complex math in littlefs. The division is effectively finding the number of blocks to iterate over, but each block has on average 2 pointers, so it doesn't work out that nicely.

Hmm, but the block_size is likely a power-of-two, and 2*4 is a power of two. Is there a shortcut for division by the sum of two powers of two?

[geky]

Pushed up a change to do this 👍 I believe the main difference is I'm using macro names without the LFS_STATICCFG_* prefix.

[tim-nordell-nimbelink]

:) Glad you liked the idea of partials.

Note that using MACROs like that loses the type safetyness of using the static inline functions, but it's probably just a stylistic thing here. There shouldn't be a difference in the compiled size between the two choices if the compiler is doing its job.

Also, I don't know what target C standard this is targeted at (C99, C11, etc), but if it was C11, you can use static asserts (https://en.cppreference.com/w/c/language/_Static_assert) for some of the configuration values statically specified.

[geky]

Note that using MACROs like that loses the type safetyness of using the static inline functions

True, though in this case they are just rearranging arguments for another function. Also, if all of the macros are specified, we wouldn't emit the lfs_cfg struct, which would cause a type error if these were inline functions.

Also, I don't know what target C standard this is targeted at (C99, C11, etc), but if it was C11, you can use static asserts

Ah, littlefs sticks to C99, and even this has been a pain point for users. Embedded IDEs move pretty slowly.

Though one nice thing with static_asserts is that you can always have a macro that switches between nothing or a static assert if available.

The main downside would be figuring out what can be static asserted, LFS_ASSERT(LFS_CFG_BLOCK_SIZE(cfg) > LFS_CFG_PROG_SIZE(cfg)) may or may not be static assertable depending on what's defined. And it risks duplicating a lot of these checks.

[geky]

Pushed up the changes suggested by @tim-nordell-nimbelink

Now, instead of 1. defining LFS_STATICCFG and 2. defining all LFS_READ_SIZE, LFS_PROG_SIZE, etc. Configuration can now be made static by defining LFS_READ_SIZE, LFS_PROG_SIZE, etc. Thanks to a really large ifdef, if all configurations are provided, LFS_STATICCFG will be defined and the RAM cost fully removed.

Additionally, we can remove the ROM cost of each struct cfg member, allowing code savings when config is only partially defined, which is perhaps more common.

This moves all of the configuration logic in lfs.h, which has the nice side effect of keeping all of the decision making in the same location.

The only catch is that we need to differentiate between the cfg->*_max and *_MAX limits. To do this I've renamed the configuration *_max to *_limit. Note that these two are slightly different, with *_max indicating the maximum supported by the current driver, and *_limit the maximum supported by the specific instance of littlefs. However if you do define a *_LIMIT, I've added an override for the relevant *_MAX, since I can't think of a time where you wouldn't want to do that.