| Readme: |
===========================================
IceFileSystem by Leif Salomonsson 2007-2013
===========================================
Contents:
Design goals and priorities
Features
Technical
Files
Still to do
Future ideas
MountList
Using HDConfig
Flash
FSConfig options
Environment variables
Preallocation algorithm
Filename/comment/softlink limits
Other
Contact
No Warranty
Design goals and priorities
---------------------------
64bit disk filesystem supporting a high number of useful features,
by a design prioritised like following:
1-Reliability 2-Scalability 3-Efficiency 4-Speed
Features
--------
64bit file/partition/extent sizes (actually very close to 2^63 bytes).
No self imposed fragmentation of large files unless there is not enough
contigious space available.
All metadata on disk is checksummed.
This means any errors on disk will be detected a lot quicker.
Meta level jornalling. Journalling makes operations to disk atomic.
Hardlinks (directory and file), softlinks, notification, examine all,
record locking, file comments. Fully supported.
Supports sector sizes from 2^9 to 2^31 atleast in theory :).
Filesystem layout on disk is not dependant on a specific underlying
sector size. Filesystem may however place data on disk so that it is
optimal for the sector size used.
Supports blocksizes from 512 bytes to 32 KiB. Still works very well with
small blocksizes though.
Filesystem does not get slower for larger partitions (scales very well),
or when heavily fragmented.
No limit in # of files/dirs in partition/dirs.
Handles removable media.
Supports optional recycle dir.
Supports automatically truncated log files and optional filesystem logfile.
Wastes only 128k (and preallocates another 128k for meta space)
for filesystem administration data, regardless of partition size.
Dynamic handling of cache memory.
Memory is given back (progressivly) to system when not used.
Technical
---------
Extent based space and file data layout.
Main space allocator is based on TLSF algorithm, adapted for on-disk storage.
All metadata except extent headers are located in special extents
called pools and use a local bitmap to keep track of free meta space.
Metadata is very compact.
Implementation uses two level caching (sector + metadata).
Implementation supports preallocation to minimize fragmentation (and increase speed)
when several files are concurrently written to.
Files
-----
L/IceFileSystem
The filesystem executable.
Put in L: and/or RDB
Tools/FSFormat
Format generic/sfs/new(ice) partitions. (MUI GUI)
Note: You can still use Ambients Format, but some features
like recycledir, changelog, volumecomment, will not be selectable.
Tools/IceConfig
Configure and get info about icefs partitions (MUI GUI)
Tools/IceInfo
Shows locks and notifys on icefs volumes. (MUI)
C/IceCheck
Does a consistency check of the on-disk fs layout
DRIVE/A - Name of partition or volume.
Note that for now passing partition name will _not_
make sure data is flushed to drive before check, volume
name will.
QUICK/S - Only do basic check
EXTLIST/S - List extents
FREELIST/S - Show free extent linkage
DEBUG/S - Turns on additional (slow) checks useful when
debugging filesystem.
C/LogFileCtrl
Create/change/query log files.
FILENAME/A - Name of logfile
SETSIZE/S - Create logfile with size if it didn't exist,
else change size of logfile.
GETSIZE/S - Obtain size of logfile (not same as filesize).
CLEAR/S - Clear logfile (filesize will become 0).
MB/N - Specify size of logfile in MiB.
QUIET/S - Be quiet.
FORCE/S - Creation can overwrite non logfile.
C/IceSalv
Simple file salvage tool.
FROM/A - Partition or volume
TO/A - Any directory
VERBOSE/S - Display some information while salvaging
C/ShowLinks
Shows hard/soft links. For hard links it requires
filesystem that understands NewReadLink().
DIR/A - Directory
ALL/S - Scan also subfolders
Docs/icefs.readme
This file.
Docs/icefs.history
Changes over time.
Docs/icefsXXX.notes
Notes specific to a particular release.
Devs/job.device
Device used by FSFormat, put it in DEVS:
Mountlist
---------
Example minimal mountlist using a ram based disk:
Device = vdisk.device
Unit = 0
BlockSize = 512
Surfaces = 25
BlocksPerTrack = 32
Interleave = 0
LowCyl = 0
HighCyl = 63 /* 50 MiB */
Buffers = 8
DosType = 0x49434502
Activate = 1
Handler = L:IceFileSystem
Setting "HighCyl = 0" will make FS autodetect number of cylinders.
The Control field supports following optional arguments:
CACHELINESIZE/N -> keep atleast 4k
MAXCACHELINES/N -> same as numbuffers, overrides it.
MAXMETACACHESIZE/N -> keep atleast 16k
PREALLOCFACTOR/N -> 1...x
INACTIVETIMEOUT/N -> in 1/10 seconds
ACTIVETIMEOUT/N -> in 1/10 seconds
MAXRECYCLEDFILES/N -> 0 = no recycling..
MAXRECYCLEDBYTES/N -> 0 = no limit..
DEFAULTFILEEBIT/K -> YES/NO
LOGEVENTS/K -> YES/NO
IOALIGNMODE/N -> 0:block (default), 1:cacheline
Control field arguments override disk based settings, if available.
Using HDConfig
--------------
HDConfig does not know about IceFileSystem, so in "Edit partition" window,
use Filesystem: "Custom" and ID: "MorphOS_0x49434502". The "MorphOS_" part
tells MorphOS to load filesystem from L:.
On Pegasos/Efika the filesystem can be loaded into the RDB:
Load into RDB like this:
Click on the magnifying glass next to the harddrive in HDConfig
main window. Select "Edit filesystems.." then "Add.." (add L:IceFileSystem
here) then insert "2" and "9" for version,revision fields. Let priority,
stacksize and patchflags be what they where. Set "Filesystem: Custom" and
put "0x49434502" in string gadget. Click "Ok" and Save changes.
On Mac's, RDB can not be used for the boot harddrive, but any additional
drives should work.
Flash
-----
To use Flash based drives optimally, in IceConfig, select "128K"
(or whatever is the native erase-size) as Cacheline size and set
IO align mode to "Cacheline". Any blocksize should work well,
say between 512 and 4096. Bigger is not really faster.
Increase inactive and active timeouts using IceConfig
to 5s and 20s, respectively.
Make sure Prealloc value is atleast 128.
Note that IO align mode currently is not changeable runtime.
(will take effect next start of filesystem)
Note that for USB 1.1 and/or G3 machines, using smaller
cachelines might still work better, even if it means driver/hardware
must take care of the actual alignment.
FSConfig
--------
The FSConfig tool allows you to change settings of a filesystem on a
partition basis. Settings for a particular partition will remain even
if partition is formatted.
Options:
"Size of cacheline"
The main cache is made up of "lines". Filesystem tries to make reads
(and writes, if ioalign mode is set to cacheline) using a full cache line,
to minimise the number of actual accesses to underlying media, which is often
quite slow in comparison to accessing a cache in memory.
"Max number of cachelines"
When this many cachelines are in use, no more will be allocated but instead
lines will be reused (and possibly flushed first, if dirty).
"Max size of meta cache"
This is how many bytes filesystem should use (at most) for caching meta data.
The meta cache is a smaller and faster cache specifically used for meta data.
Following 3 are not really options but displays some information:
"Max total cache usage"
Displays the maximum amount of memory filesystem is currently allowed to use for caching.
"Current cache usage"
Displays how much memory filesystem is currently using for caching.
"..of which is hot"
Displays how much memory in the cache is currently dirty.
"Cacheline release"
Number of seconds until an unused cacheline is automatically freed.
"Prealloc factor"
Filesystem can allocate space "ahead" when writing to a file, to minimise
fragmentation and even increase speed. Higher values means more preallocation.
See "Other" section for more information.
"Inactive update timeout"
Modified meta data is forced down to disk, if no modifications has occured
for this period of time.
"Active update timeout"
The absolute maximum time dirty meta can be kept in memory before flushed
to disk, even with modifying activity going on.
"Max recycled files"
Maximum amount of files to be kept in the .recycledir.
"0" means no files and will effectively turn off the .recycledir functionality.
Default is 128. Note that keeping several thousand files in recycle dir will
slow down deletion operations considerably.
"Max recycled bytes"
Maximum amount of space the .recycledir contents may use.
If defined, will also deny recycling of files that are larger than maximum bytes / 16.
This way one large file cannot kick out all previous files in recycledir.
Default is unlimited.
"Default file E-bit"
The (E)xecutable bit in files is NOT set by default by IceFS, check this option
if you want this behaviour.
"Log file changes"
The filesystem log (named .logfile and placed in root directory) logs modifications
to disk. Unticking this option can be used to turn off the logging.
"IO align mode"
Setting this to cacheline, makes caching system try to align all reads and writes to
cacheline boundaries. Otherwise Block alignment is used. Block alignment should be fine
for normal harddrives.
"Debug flags"
Integer value desribing debug flags for filesystem. Mainly for debugging.
Intentionally undocumented for now.
Note: Use double click or [return] on a partition to get more information about it.
Environment variables
----------------------
ICE_AUTORETRYIO - Set it to 0..9 for number of automatic retries per failed
io request, before putting up a requester. Will not come
into effect until 3 seconds after FS is started. Default is
1 automatic retries. Currently only errors in range 42..45
are retried because these are the ones that seem to pop up
on a regular basis with certain io devices (USB in particular),
and the reason for this variable to begin with.
Preallocation algorithm
-----------------------
Preallocation algorithm looks at
a) the size of the write: if exact power of 2 and atleast 512 bytes, lets preallocate.
or
b) the previous size of a file: if > 0 before appending write, lets preallocate.
The space that gets preallocated is the size of the write times the prealloc factor.
Don't stop using SetFileSize() just because filesystem can preallocate though.
If you know the final filesize, SetFileSize() is always a better choice.
If IceCheck outputs note about a file has "unchoped preallocated extent", it means
computer was resetted/crashed before application called Close() after modifying a
file which now has some preallocated space reserved for it. This is not a problem
but might use some disk space which will not come to use until file is properly
open+closed or deleted.
Filename/comment/softlink limitations
-------------------------------------
Filename length is limited by packet API to 255 characters.
It is further limited by the FileInfoBlock to 107 characters. When
FileInfoBlock is not used (like when creating/renaming/deleting a file
in shell, or when listing a file using Dir command, up to 255 characters
can be used. Ambient seems to have a 107 character limit.
Comment length is limited by packet API to 255 characters.
Again the FileInfoBlock limits this to 79 characters.
Also the FileNote shell command currently limits comment to
79 characters. Ambient seems to allow up to 106 characters
of comment.
Softlink target path length is limited to 992 characters.
Other
-----
Filesystem DOS type is 0x49434502 (this has changed since version 1.x !).
The "BufMemType" value is not needed in mountlist (it is read from the drivegeometry instead).
Uses Trackdisk64, or if not available SCSI Direct, for devices >= 2 GiB.
Maximum disk size is currently about 2TB due to API in MorphOS,
and so partitions will be limited to slightly less than this.
Special thanks to:
Fabien Coeurjoly and Guido Mersmann for testing alpha versions :)
Mark Olsen for helping out with caching system :)
Fulvio Peruggi for lengthy bug reports :)
Herbert Klackl, digiman, Kenny Dahlroth :)
Jacek�Piszczek for various :)
Patrick Jung for nice icons :)
All of those who registered ! :)
Contact
-------
dev blubbedev net
http://blubbedev.net/icefs
No Warranty
-----------
There may be bugs or unforeseen interactions.
Use this software at your own risk.
|
Ambient (143)
ICE-FS_2.12.lha