Add LittleFS as an optional filesystem, API compatible w/SPIFFS (but not on-flash-format compatible) #5511
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
earlephilhower
changed the title
devyte
requested changes
Dec 17, 2018
There was a problem hiding this comment.
I find this idea very interesting. It seems to not have the SPIFFS performance issues, and also has real dir support, which are the two main sticking points with spiffs at the moment.
I suggest proceeding with this as an experimental alternative to SPIFFS and get exposure to users, so that testing and feedback can happen asap.
|
Example output from the sample showing from ~3x to ~24x(!!) faster LittleFS operation (YMMV, depends on flash chip and access pattern): |
earlephilhower
force-pushed
the
littlefs
branch
from
64a2472 to
2dad177
Compare
Adds a LittleFS object which uses the ARMmbed littlefs embedded filesystem, https://github.com/ARMmbed/littlefs, to enable a new filesystem for onboard flash utilizing the exact same API as the existing SPIFFS filesystem. LittleFS is built for low memory systems that are subject to random power losses, is actively supported by the ARMmbed community, supports directories, and seems to be much faster in the large-ish read-mostly applications I use. LittleFS, however, has a larger minimum file allocation unit and does not do static wear levelling. This means that for systems that need many little files (<4K), have small SPIFFS areas (64K), or which have a large static set of files covering the majority of flash coupled with a frequently updated set of other files, it may not perform as well. Simply replace SPIFFS.begin() with LittleFS.begin() in your sketch, use LittleFS.open in place of SPIFFS.open to open files, and everything else just works thanks to the magic of @igrr's File base class. **LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS** Since it is a completely different filesystem, you will need to reformat your flash (and lose any data therein) to use it. Tools to build the flash filesystem and upload are at https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and https://github.com/earlephilhower/mklittlefs/ . The mklittlefs tool is installed as part of the Arduino platform installation, automatically. The included example shows a contrived read-mostly example and demonstrates how the same calls work on either SPIFFS.* or LittleFS.* Host tests are also included as part of CI. Directories are fully supported in LittleFS. This means that LittleFS will have a slight difference vs. SPIFFS when you use LittleFS.openDir()/Dir.next(). On SPIFFS dir.next() will return all filesystem entries, including ones in "subdirs" (because in SPIFFS there are no subdirs and "/" is the same as any other character in a filename). On LittleFS, dir.next() will only return entries in the directory specified, not subdirs. So to list files in "/subdir/..." you need to actually openDir("/subdir") and use Dir.next() to parse through just those elements. The returned filenames also only have the filename returned, not full paths. So on a FS with "/a/1", "/a/2" when you do openDir("/a"); dir.next().getName(); you get "1" and "2" and not "/a/1" and "/a/2" like in SPIFFS. This is consistent with POSIX ideas about reading directories and more natural for a FS. Most code will not be affected by this, but if you depend on openDir/Dir.next() you need to be aware of it. Corresponding ::mkdir, ::rmdir, ::isDirectory, ::isFile, ::openNextFile, and ::rewind methods added to Filesystem objects. Documentation has been updated with this and other LittleFS information. Subdirectories are made silently when they do not exist when you try and create a file in a subdir. They are silently removed when the last file in them is deleted. This is consistent with what SPIFFS does but is obviously not normal POSIX behavior. Since there has never been a "FS.mkdir()" method this is the only way to be compatible with legacy SPIFFS code. SPIFFS code has been refactored to pull out common flash_hal_* ops and placed in its own namespace, like LittleFS.
earlephilhower
force-pushed
the
littlefs
branch
from
680d043 to
cd03a6f
Compare
6 tasks
Enable setConfig for LittleFS as well plys merge the SPIFFS changes done in the SDFS PR.
6 tasks
|
Looks like V2 alpha is one step closer to committal. littlefs-project/littlefs#85 (comment) |
earlephilhower
force-pushed
the
littlefs
branch
from
f5c6d51 to
e3e8de3
Compare
earlephilhower
changed the title
The V2-alpha branch supports small file optimizations which can help increase the utilization of flash when small files are prevalent. It also adds support for metadata, which means we can start adding things like file creation times, if desired (not yet).
|
Updated to v2-alpha branch of LittleFS, but I think I'm running into a bug with small files. We'll see what happens...littlefs-project/littlefs#85 (comment) |
6 tasks
In a directory, the order in which "readNextFile()" will return a name is undefined. SPIFFS may return it in order, but LittleFS does not as of V2. Update the test to look for files by name when doing readNextFile() testing.
|
V2 now works and passes host tests. Now to fix the 10 file merge conflicts.... |
…lefs Everything compiles, but host tests do not yet pass. Add LittleFS.truncate() method. Add SDFS test to cover all SPIFFS/LittleFS ones.
TD-er
suggested changes
Apr 18, 2019
Thanks to @TD-er for discovering the issue
|
Really looking forward to built-in LittleFS support, any May updates? |
|
We've got a bugfix only rev due any time now, but thien this is scheduled for merge. Give it a couple weeks. You can always pull the PR and use it yourself to test things. |
ascillato
added a commit
to ascillato/Tasmota_KNX
that referenced
this pull request
May 30, 2019
The actual Stage ESP8266 Core of Arduino (next 2.6.0) had changed the SPIFFS defines of the memory to FS due to a change in the libraries (esp8266/Arduino#5511)
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a
LittleFSobject which uses the ARMmbed littlefs to enable a new filesystem for onboard flash, utilizing the exact same API as the existing SPIFFS filesystem. LittleFS is built for low memory systems that are subject to random power losses, is actively supported by the ARMmbed community, supports directories (not implemented in this PR to minimize the SPIFFS underlying FS class changes), and seems to be much faster in the large-ish read-mostly applications I use.Simply replace
SPIFFS.begin()withLittleFS.begin()in your sketch, useLittleFS.openin place ofSPIFFS.opento open files, and everything else just works thanks to the magic of @igrr'sFilebase class.LITTLEFS FLASH LAYOUT IS INCOMPATIBLE WITH SPIFFS. Since it is a completely different filesystem, you will need to reformat your flash (and lose any data therein) to use it. Tools to build the flash filesystem and upload are at https://github.com/earlephilhower/arduino-esp8266littlefs-plugin and https://github.com/earlephilhower/mklittlefs/
While SPIFFS seems to work well for small files accessed infrequently, its performance can be very slow on large files when doing random seeking and updates. The included example shows a contrived read-mostly example and demonstrates how the same calls work on either SPIFFS.* or LittleFS.*
Toolchain integration is provided by mklittlefs and arduino-plugin-littlefs.
May be of interest in #4291, #4748, and others.