LibrePilot/flight/Libraries/dosfs

README.TXT                                         (C) Copyright 2006
DOSFS Level 1 Version 1.02      Lewin A.R.W. Edwards (sysadm@zws.com)
=====================================================================

Abstract
========
DOSFS is a FAT-compatible filesystem intended for fairly low-end
embedded applications. It is not the leanest possible implementation
(the leanest FAT implementations operate in << 512 bytes of RAM, with
heavy restrictions). This code strikes a good balance between size
and functionality, with an emphasis on RAM footprint.

Intended target systems would be in the ballpark of 1K RAM, 4K ROM
or more.

Features:
* Supports FAT12, FAT16 and FAT32 volumes
* Supports storage devices up to 2048Gbytes in size (LBA32)
* Supports devices with or without MBRs (hard disks vs. floppy disks
  or ZIP drives formatted as "big floppies")
* Supports multiple partitions on disks with MBRs
* Supports subdirectories
* Can be operated with a single global 512-byte sector buffer
* Fully reentrant code (assuming the underlying physical device driver
  is reentrant and global sector buffers are not used). There are no
  global variables in the filesystem
* Does not perform any memory allocation
* Partial support for random-access files

Applications:
* Firmware upgrades
* Failsafe IPL
* Media playback
* Data logging
* Configuration storage

There is no technical support for this free product; however, if you
have questions or suggestions, you are encouraged to email Lewin
Edwards at sysadm@zws.com. If you need custom additions to the code,
or if you have other projects for which you need engineering
assistance, please feel free to email or call (646) 549-3715.

License
=======
The license for DOSFS is very simple but verbose to state.

1. DOSFS is (C) Copyright 2006 by Lewin A.R.W. Edwards ("Author").
   All rights not explicitly granted herein are reserved. The DOSFS
   code is the permanent property of the Author and no transfer of
   ownership is implied by this license.

2. DOSFS is an educational project, provided as-is. No guarantee of
   performance or suitability for any application is stated or
   implied. You use this product entirely at your own risk. Use of
   this product in any manner automatically waives any right to seek
   compensation or damages of any sort from the Author. Since the
   products you might make are entirely out of the Author's control,
   use of this product also constitutes an agreement by you to take
   full responsibility for and indemnify the Author against any
   action for any loss or damage (including economic loss of any
   type, and specifically including patent litigation) that arises
   from a product made by you that incorporates any portion of
   the DOSFS code.

3. If you live under the jurisdiction of any legislation that would
   prohibit or limit any condition in this license, you cannot be
   licensed to use this product.

4. If you do not fall into the excluded category in point 3, you are
   hereby licensed to use the DOSFS code in any application that you
   see fit. You are not required to pay any fee or notify the Author
   that you are using DOSFS. Any modifications made by you to the
   DOSFS code are your property and you may distribute the modified
   version in any manner that you wish. You are not required to
   disclose sourcecode to such modifications, either to the Author or
   to any third party. Any such disclosure made to the Author will
   irrevocably become the property of the Author in the absence of a
   formal agreement to the contrary, established prior to such
   disclosure being made.

To summarize the intent of the above: DOSFS is free. You can do what
you want with it. Anything that happens as a result is entirely your
responsibility. You can't take ownership of my code and stop me from
doing whatever I want with it. If you do something nifty with DOSFS
and send me the sourcecode, I may include your changes in the next
distribution and it will be released to the world as free software.
If someone sues you because your DOSFS-containing product causes
any sort of legal, financial or other problem, it's your lawsuit,
not mine, and you'll exclude me from the proceedings.

User-Supplied Functions
=======================
You must provide functions to read sectors into memory and write
them back to the target media. The demo suite includes an emulation
module that reads/writes a disk image file (#define HOSTVER pulls
in hostemu.h which wraps the prototypes for these functions).
There are various tools for UNIX, DOS, Windows et al, to create
images from storage media; my preferred utility is dd.

The functions you must supply in your embedded app are:

DFS_ReadSector(unit,buffer,sector,count)
DFS_WriteSector(unit,buffer,sector,count)

These two functions read and write, respectively, "count" sectors of
size SECTOR_SIZE (512 bytes; see below) from/to physical sector
#"sector" of device "unit", to/from the scratch buffer "buffer". They
should return 0 for success or nonzero for failure. In the current
implementation of DOSFS, count will always be 1.

The "unit" argument is designed to permit implementation of multiple
storage devices, for example multiple media slots on a single device,
or to differentiate between master and slave devices on an ATAPI bus.
                 
This code is designed for 512-byte sectors. Although the sector size
is a #define, you should not tinker with it because the vast majority
of FAT filesystems use 512-byte sectors, and the DOSFS code doesn't
support runtime determination of sector size. This will not affect the
vast majority of users.

Example Code
============
Refer to the tests in main.c to see how to call DOSFS functions.
(These tests are all commented out). Note that the only two files
you need to add to your project are dosfs.c and dosfs.h.


Mounting Volumes
================
--If the device has a partition table (practically all removable flash
  media are formatted this way), call DFS_GetPtnStart to get the
  starting sector# of the desired partition. You can optionally also
  retrieve the active state, partition type byte and partition size
  in this step. The reason this step is broken out separately is so
  you can support devices that are formatted like a floppy disk, i.e.
  the volume starts directly at physical sector 0 of the media.

--Call DFS_GetVolInfo to read filesystem info into a VOLINFO structure.
  DFS_GetVolInfo needs to know the unit number and partition starting
  sector (as returned by DFS_GetPtnStart, or 0 if this is a "floppy-
  format" volume without an MBR).

From this point on, the VOLINFO structure is all you'll need - you can
forget the unit and partition start sector numbers.

Enumerating Directory Contents
==============================
--Call DFS_Opendir and supply a path, populated VOLINFO and a
  DIRINFO structure to receive the results. Note - you must PREPOPULATE
  the DIRINFO.scratch field with a pointer to a sector scratch buffer.
  This buffer must remain unmolested while you have the directory open
  for searching.
--Call DFS_GetNext to receive the DIRENT contents for the next directory
  item. This function returns DFS_OK for no error, and DFS_EOF if there
  are no more entries in the directory being searched.
  Before using the DIRENT, check the first character of the name. If it
  is NULL, then this is an unusable entry - call DFS_GetNext again to
  keep searching. LFN directory entries are automatically tagged this way
  so your application will not be pestered by them.

  Note: A designed side-effect of this code is that when you locate the
  file of interest, the DIRINFO.currentcluster, DIRINFO.currentsector
  and DIRINFO.currententry-1 fields will identify the directory entry of
  interest.

Reading a File
==============
--Call DFS_OpenFile with mode = DFS_READ and supply a path and the relevant
  VOLINFO structure. DFS_OpenFile will populate a FILEINFO that can be used
  to refer to the file.
--Optionally call DFS_Seek to set the file pointer. If you attempt to set
  the file pointer past the end of file, the file will NOT be extended. Check
  the FILEINFO.pointer value after DFS_Seek to verify that the pointer is
  where you expect it to be.
--Observe that functionality similar to the "whence" parameter of fseek() can
  be obtained by using simple arithmetic on the FILEINFO.pointer and
  FILEINFO.filelen members.
--Call DFS_ReadFile with the FILEINFO you obtained from OpenFile, and a
  pointer to a buffer plus the desired number of bytes to read, and a
  pointer to a sector-sized scratch buffer. The reason a scratch sector is
  required is because the underlying sector read function doesn't know
  about partial reads.
--Note that a file opened for reading cannot be written. If you need r/w
  access, open with mode = DFS_WRITE (see below).

Writing a file
==============
--Call DFS_OpenFile with mode = DFS_WRITE and supply a path and the relevant
  VOLINFO structure. DFS_OpenFile will populate a FILEINFO that can be used to
  refer to the file.
--Optionally call DFS_Seek to set the file pointer. Refer to the notes on
  this topic in the section on reading files, above.
--Call DFS_WriteFile with the FILEINFO you obtained from OpenFile, and a
  pointer to the source buffer, and a pointer to a sector-sized scratch
  buffer.
--Note that a file open for writing can also be read.
--Files are created automatically if they do not exist. Subdirectories are
  NOT automatically created.
--If you open an existing file for writing, the file pointer will start at
  the beginning of the data; if you want to append, seek to the end before
  writing new data.
--If you perform random-access writes to a file, the length will NOT change
  unless you exceed the file's original length. There is currently no
  function to truncate a file at the current pointer position.
--On-disk consistency is guaranteed when DFS_WriteFile exits, unless your
  physical layer has a writeback cache in it.

Deleting a file
===============
--Call DFS_UnlinkFile
--WARNING: This call will delete a subdirectory (correctly) but will NOT
  first recurse the directory to delete the contents - so you will end up
  with lost clusters.

Notes
=====
Some platforms may require explicit pragmas or attributes to the structures
and unions. For example, arm-gcc will require __attribute__ ((__packed__))
otherwise it will try to be "smart" and place the uint8_t members on 4-byte
boundaries. There is no truly elegant compiler-independent method to get
around this sort of problem.

The code assumes either a von Neumann architecture, or a compiler that
is smart enough to understand where your pointers are aimed and emit
the right kind of memory read and write instructions. The implications
of this statement depend on your target processor and the compiler you
are using. Be very careful not to straddle bank boundaries on bank-
switched memory systems.

Physical 32-bit sector numbers are used throughout. Therefore, the
CHS geometry (if any) of the storage media is not known to DOSFS. Your
sector r/w functions may need to query the CHS geometry and perform
mapping.

File timestamps set by DOSFS are always 1:01:00am on Jan 1, 2006. If
your system has a concept of real time, you can enhance this.

FILEINFO structures contain a pointer to the corresponding VOLINFO
used to open the file, mainly in order to avoid mixups but also to
obviate the need for an extra parameter to every file read/write. DOSFS
assumes that the VOLINFO won't move around. If you need to move or
destroy VOLINFOs pertaining to open files, you'll have to fix up the
pointer in the FILEINFO structure yourself.

The subdirectory delimiter is a forward slash ( '/' ) by default. The
reason for this is to avoid the common programming error of forgetting
that backslash is an escape character in C strings; i.e. "\MYDIR\FILE"
is NOT what you want; "\\MYDIR\\FILE" is what you wanted to type. If you
are porting DOS code into an embedded environment, feel free to change
this #define.

DOSFS does not have a concept of "current directory". A current directory
is owned by a process, and a process is an operating system concept.
DOSFS is a filesystem library, not an operating system. Therefore, any
path you provide to a DOSFS call is assumed to be relative to the root of
the volume.

There is no call to close a file or directory that is open for reading or
writing. You can simply destroy or reuse the data structures allocated for
that operation; there is no internal state in DOSFS so no cleanup is
necessary. Similarly, there is no call to close a file that is open for
writing. (Observe that dosfs.c has no global variables. All state information
is stored in data structures provided by the caller).

MAX_PATH is defined as 64. MS-type DOS filesystems support 128 characters
or more in paths. You can increase this define, but it may GREATLY
increase memory requirements.

VFAT long filenames are not supported. There is a certain amount of
patent controversy about them, but more importantly they don't really
belong in the scope of a "minimalist embedded filesystem".

Improving Performance
=====================
Read performance is fairly good, but can be improved by implementing read
caching on the FAT (see below) and, depending on your hardware platform,
possibly by implementing multi-sector reads.

Write performance may benefit ENORMOUSLY from platform-specific
optimization, especially if you are working with a flash media type that
has a large erase block size. While it is not possible to offer detailed
platform-independent advice, my general advice is to implement writeback
caching on the FAT area. One method for doing this would be to have a
cache system that lives in the DFS_ReadSector/WriteSector functions (on
top of the physical sector r/w functions) and is initially switched off.
Once you have called DFS_GetVolInfo, you then extract the VOLINFO.fat1
and VOLINFO.rootdir parameters and pass them to your caching layer.
Sectors >= fat1 and < rootdir should be cached. The cache strategy is
determined by the physical storage medium underlying the filesystem.

CACHING HINT:
Observe that there will be numerous read-modify-write operations in the
region from VOLINFO.fat1 through VOLINFO.fat1+VOLINFO.secperfat-1, but
in the region from VOLINFO.fat1+VOLINFO.secperfat through VOLINFO.rootdir
there will ONLY be write operations.

Platform Compatibility
======================
DOSFS was derived from code originally written for ARM7TDMI but
designed to be portable. It has been tested on AVR (using avrgcc),
MSP430 (using Rowley's CrossWorks) and PPC603e (using gcc); the host
test suite has also been validated on x86 using gcc under both Cygwin
and 32-bit Fedora Core 4 Linux.

TODO list
=========
* Add function to create subdirectory
* Make DFS_UnlinkFile recognize non-empty subdirectories
* Support "fast write" files where the FAT is not updated, for
  logging applications where latency is important.

Test cases for V1.02
====================
Version 1.02 has NOT been through full regression testing. However the
bugs fixed in this version are important, and people have been asking
about them.

Test cases for V1.01
====================
See below.

Test cases for V1.00
====================
These are the test cases that were used to validate the correct
functionality of the DOSFS suite. Each test was performed on FAT12,
FAT16 and FAT32 volumes. P=Pass, F=Fail.

Case                                                      F12 F16 F32
---------------------------------------------------------------------
Get volume information                                    P   P   P
Open root directory                                       P   P   P
List contents of root directory (fully populated)         P   P   P
Open subdirectory                                         P   P   P
List contents of subdirectory (<= 1 cluster)              P   P   P
List contents of large subdirectory (> 1 cluster)         P   P   P
Open 5-level nested subdirectory                          P   P   P
Open existing file for reading                            P   P   P
Open nonexistent file for reading                         P   P   P
Seek past EOF, file open for reading                      P   P   P
Seek to cluster boundary                                  P   P   P
Seek past cluster boundary                                P   P   P
Seek backwards to nonzero offset, pointer > cluster size  P   P   P
Block-read entire file >1 cluster in size, odd size       P   P   P
Seek to odd location in file                              P   P   P
Perform <1 sector reads from random file locations        P   P   P
Open nonexistent file for writing in root dir             P   P   P
Open nonexistent file for writing in subdir               P   P   P
Repeat prev. 2 tests on volume with 0 free clusters       P   P   P
Seek past EOF, file open for writing                      P   P   P
Open existing file for writing in root dir                P   P   P
Write random-length records to file, 20 clusters total    P   P   P
MS-DOS 6.0 SCANDISK cross-check                           P   P   P

Revision History
================
Jan-06-2005 larwe Initial release (1.0)
Jan-29-2006 larwe Bugfix release (1.01)
 - Fixed error in FAT12 FAT read on boundary of sector
 - Improved compilability under avrgcc
Sep-16-2006 larwe Bugfix release (1.02)
 - DFS_Seek would not correctly rewind to start of file
 - DFS_Seek would not correctly seek to a position not on a cluster
   boundary
 - DFS_OpenFile fencepost error caused memory access at [start of
   string-1] with a local variable
 - DFS_OpenFile could not open a file in the root directory