Whats an optimal way to build a filesystem and flash it?

[TobyChaloner]

I wish to construct and upload a filesystem to a target. Initially I'm looking at the building of the filesystem image.
I'm trying to follow the Arduino IDE concept of having a data/ directory from the sketch directory. The IDE builds an image using a plugin.
I notice on the ESP32 v2.0.3 that it mandates a partition table (via recipe.objcopy.partitions.bin.pattern=...), but there is no obvious detection of a data directory and it contents; and subsequent building. The ESP8266 v3.1.2 has no apparent need for a partition table.
My initial thought is to either have directives in the sketch CMakeLists.txt or to have a platform.local.txt. I'm not keen on the platform.local.txt solution as its not in the same version controlled place as the sketch.
Ideally I'd like to go with mksmallfs, but mkspliffs appears more mature, so may be useful in the first instance.
Is there an effective means to identify the board specific information read from boards.txt, which provides information like pagesize and start / end etc

My clunky CMakeLists.txt looks like

set (FS_PAGE_SZ          256) # bytes
set (FS_BLK_SZ          4096) # FILESYSTEM_BLOCK_SIZE bytes
set (FS_START_ADDR   0xEB000) #FS start addr 0xEB000
set (USB_PORT /dev/ttyUSB0) # probably best presented at the command line TODO

file(COPY data DESTINATION .)

# make a dependency on the files to rebuild the filesystem.bin
file(GLOB_RECURSE filesystem_files
  LIST_DIRECTORIES true
  RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}
  CONFIGURE_DEPENDS
  data/*)

#
#TODO not sure if this is run every compile time
#
list(JOIN filesystem_files " " filesystem_files) # turn the list from ; separated into a space separated list

add_custom_command(OUTPUT filesystem.dep
  COMMAND echo  "filesystem.bin: ${filesystem_files}" > filesystem.dep  
  COMMENT "Populating filesystem.dep"
  VERBATIM
  )

add_custom_command(OUTPUT filesystem.bin
  COMMAND  ${ARDUINO_BOARD_RUNTIME_PLATFORM_PATH}/../../../tools/mklittlefs/3.1.0-gcc10.3-e5f9fec/mklittlefs
  --create data --block ${FS_BLK_SZ} --page ${FS_PAGE_SZ}
  filesystem.bin
  DEPFILE filesystem.dep
  DEPENDS filesystem.dep
  COMMENT "Building Filesystem image"
  VERBATIM)


if (ARDUINO)
  add_custom_target(upload-fs
    python ${ARDUINO_BOARD_RUNTIME_PLATFORM_PATH}/tools/esptool/esptool.py --port  ${USB_PORT} write_flash ${FS_START_ADDR} filesystem.bin
    DEPENDS filesystem.bin
    COMMENT "Uploading") 
endif ()


#
#use to check contents of the filesystem are valid and what they are
#
add_custom_target(debug-list-fs
  ${ARDUINO_BOARD_RUNTIME_PLATFORM_PATH}/../../../tools/mklittlefs/3.1.0-gcc10.3-e5f9fec/mklittlefs --list filesystem.bin
  DEPENDS filesystem.bin
  COMMENT "*** Showing hierachy of files in the filesystem ***") 

[TobyChaloner]

Adding new function to BoardBuildTargets.cmake

This would entail following the pattern of function(target_enable_arduino_upload target) using new rules defined in platform.txt

platform.local.txt

tools.mklittlefs.path={runtime.tools.mklittlefs.path}
tools.mklittlefs.data_dir=data
tools.mklittlefs.fs_image_name=littlefs.bin
tools.mklittlefs.filesystem.pattern={tools.mklittlefs.path}/{tools.mklittlefs.cmd} --create {tools.mklittlefs.data_dir}  --block {build.spiffs_blocksize} --page {build.spiffs_pagesize} "{build.path}/{fs_image_name}"
tools.esptool.upload.littlefs.pattern="{cmd}" -I "{runtime.platform.path}/tools/upload.py" --chip esp8266 --port "{serial.port}" --baud "{upload.speed}" "{upload.verbose}" {upload.resetmethod} write_flash {build.spiffs_start} "{build.path}/{fs_image_name}"

Drawbacks: the turnaround time for an update to the platform.txt is not likely to be quick.