Removable media/memory card information

[uweseimet]

Sometimes users are unsure about how to configure HDDRIVER's removable media settings. Let's have a look at some details: How does a drive report a media change, how does a driver have to react, and which removable media/memory card settings does HDDRIVER offer? Whether a drive supports removable media is displayed in HDDRUTIL's "Device Information" dialog.

Essentially, according to the IDE/SATA and SCSI specifications a drive has to report a media change when it receives a command which is affected by the change. Such a command is usually the first read or write operation after the media change. Instead of executing this operation the drive returns a special error code, which notifies the driver about the change. The driver has to react, e.g. by flushing caches, and then re-sending the previous command if it is still applicable.
If a drive does not properly report the media change, or if a driver does not react correctly, a loss of data is likely and a hot-swap of removable media without a reboot cannot be supported. This is because TOS always caches data in the so-called GEMDOS cache. If this cache is not cleared after a media change, the data on the new medium will likely be overwritten by stale data from the previous medium. This data corruption you might not notice immediately. There may also be errors, e.g. crashes, when just reading but not writing data, because there may still be stale data from the previous medium in the GEMDOS cache.

Real removable media drives, e.g. SyQuest, ZIP/JAZ, UltraSatan, GigaFile, and drive emulations like SCSI2Pi are known to deal correctly with media changes. Some low-cost ACSI SD card solutions do not follow the specifications, so that media changes can result in the loss/corruption of data. You can verify whether your drive correctly reports a media change with the MCHGTEST tool, which is included in the SCSI Driver test suite.

Never remove a medium while it is being accessed, which is sometimes indicated by an access LED. With HDDRIVER, the CHK_OFLS tool locks the eject button of removable media drives as long as there are still open files. The "Unlock after Reset" option is kind of a precaution that unlocks the medium when you reset the Atari, so that an eject button does not stay blocked after a reset. Memory card readers do not support pyhsically locking a medium. It is up to the user to ensure that the medium is present as long as required. With TOS and MagiC (but not with MiNT) you may assume that it is save to remove a medium when you are on the desktop and do not access the drive, because TOS and MagiC do not postpone read or write operations.

With respect to the operating system or driver, additional actions may have to be taken after a media change. The AHDI Release Notes, which AHDI compatible drivers have to comply with, define what has to be done, in particular when a new medium contains more or less partitions than the medium that was present when launching the driver (usually when booting). They also address the case where a partition's logical sector size on a new medium is bigger than the biggest sector size found on the boot medium.
AHDI on launch reserves at least 4 drive IDs for any removable media drive. If a medium has more than 4 partitions, as many additional IDs as required are reserved for this drive. AHDI assumes that the sector sizes of the partitions on a new medium are never bigger than those found when booting. Otherwise the affected partitions are not accessible. Up to a certain degree this behavior is configurable, but AHDI in contrast to HDDRIVER does not offer a convenient way to do so. The AHDI Release Notes provide more details.

It is important to realize that there is no good alternative to reserving a fixed number of drive IDs for each removable media drive. Otherwise you would run into troubles with more than one such drive, e.g. a memory card reader with several slots, like the UltraSatan. Without a fixed number of IDs there would also be issues with just a single removable media drive and additional regular drives with higher device IDs.
As an example, assume that your first drive is a removable media drive with 2 partitions, and there is also a second drive. The IDs C: and D: are assigned to the 2 partitions on the removable media drive, E: and F: are reserved by the default settings (4 partitions on removable media), and G: and onwards are assigned to the second drive. When you later insert a medium with 4 partitions, these will be assigned the IDs C:, D:, E: and F:, using all of the previously reserved 4 IDs. The drive IDs on the second drive remain unchanged, regardless of the number of partitions on the new removable medium. Without reserving IDs, the IDs on the second drive would change (be shifted) whenever a medium with a different number of partitions than before is inserted. Shifting IDs would result in a dangerous mess: The paths of some files might change, applications that remember path settings would not work anymore etc. Having a fixed number of reserved drive IDs per removable media drive ensures that existing drive ID assignments always remain valid and predictable. There is a disadvantage, though: If you insert a medium with 5 partitions, the fifth partition is not accessible when only 4 drive IDs have been reserved. See below how to address this with HDDRIVER.

There is more to consider when dealing with removable media, namely the sector sizes. These depend on the partition sizes: The bigger a partition, the bigger its logical sector size. If after a media change one of the partitions on the new medium has a bigger logical sector size than any of the partitions present when booting, this partition will not be accessible. This is because at boot time memory was allocated for the biggest known sector, and TOS does not allow to change this memory allocation later. This typically happens when there are only small partitions when booting and you later insert a medium with bigger partitions. This problem can be resolved with HDDRIVER, see below.

Everything explained so far applies to any kind of drive or AHDI compatible driver. HDDRIVER is fully compliant with the AHDI specification, but you can configure it to address the issues mentioned above without compromising AHDI compatibility. Have a look at HDDRUTIL's "Removable Media/Memory Cards" settings. There you can configure the number of drive IDs to initially reserve for removable media drives. This number should be the maximum number of partitions on any of your media. This ensures that you will always have access to all partitions, even when less partitions were present on your removable medium when booting than you need later. When you configure HDDRIVER to reserve 0 partitions, you can only access as many partitions per removable media drive as were present for this drive when booting.
Note that there is no need to press the ESC key after changing a medium. As long as the drive properly reports the change, HDDRIVER and TOS deal with everything automatically. Remember, some low cost ACSI memory card solutions may not correctly report media changes.

With HDDRIVER you can also configure the maximum sector size to reserve memory for. This addresses the problem of only having small partitions when booting and later inserting a medium with bigger partitions. In order to help with configuring this setting, HDDRUTIL displays the current biggest sector size in the "Removable Media/Memory Cards" dialog. If you are unsure you can simply set this value to 32768, which is the maximum logical sector size supported by any version of TOS. If the configured value is higher than required by your actual set of partitions you permanently use more memory than actually needed, but this will not do any harm besides wasting some memory. But note: If you increase the cache sizes (see below), the amount of memory wasted will also increase.

In HDDRUTIL's basic settings you can configure the cache mentioned above. HDDRIVER supports two caches, which are not proprietary but are the standard GEMDOS caches. The minimum cache size imposed by TOS is 2 sectors per cache, i.e. the GEMDOS cache cannot be switched off. If you multiply the cache size with the size of the biggest logical sector, you get the total amount of memory allocated by the cache. Remember that this memory is allocated when booting, i.e. the amount of memory needed depends on the partition sizes of the media present when booting. HDDRIVER display the amount of memory used in its boot message. You can boot from a removable medium just like you can boot from a hard drive. There are no special restrictions on the the boot drive's partition size. Any size supported by your version of TOS is fine.
If you configure HDDRUTIL to install HDDRIVER in Fast-RAM, the memory used for the caches also resides in Fast-RAM, which provides a performance benefit and saves precious ST-RAM. Ataris with Fast-RAM and IDE hardware extension profit even more, because when running HDDRIVER in Fast-RAM there is a considerable throughput improvement with IDE transfers
Re-install HDDRIVER after changing the Fast-RAM configuration setting in order for the change to become effective. In case you are using HDDRIVER modules these run in the same kind of RAM HDDRIVER was configured to run in.

Remember: All of this information is not only relevant for old-style removable media drives but also for SCSI device emulations and memory cards. There is a rare exception, though: Some industrial grade CF cards (on purpose) do not report themselves as being removable, or they can be configured on how to behave ("True IDE" mode). Such a card behaves like a regular hard drive, and HDDRUTIL's device information reports it as not removable. These cards are not hot-swappable, of course, because the hard disk driver does not know that they are removable.