<?xml version="1.0"?>

<document>
  <properties>
    <title>Indexed Disk Auxiliary Cache</title>
    <author email="ASmuts@yahoo.com">Aaron Smuts</author>
  </properties>

  <body>
    <section name="Indexed Disk Auxiliary Cache">
      <p> 
        The Indexed Disk Auxiliary Cache is an optional plugin for the
        JCS.  It is primarily intended to provide a secondary store to 
        ease the memory burden of the cache.  When the memory cache 
  	    exceeds its maximum size it tells the cache hub that the item
	      to be removed from memory should be spooled to disk.  The cache 
	      checks to see if any auxiliaries of type "disk" have been 
	      configured for the region.  If the "Indexed Disk Auxiliary Cache"  
	      is used, the item will be spooled to disk.
      </p>

      <subsection name="Disk Indexing">
      <p>
        The Indexed Disk Auxiliary Cache follows the fastest 
    	  pattern of disk caching.  Items are stored at the end of a file 
	      dedicated to the cache region.  The first byte of each disk entry
	      specifies the length of the entry.  The start position in the file
	      is saved in memory, referenced by the item's key.  Though this still
	      requires memory, it is insignificant given the performance trade 
	      off.  Depending on the key size, 500,000 disk entries will probably 
  	    only require about 1 MB of memory.  Locating the position of an item is 
	      as fast as a map lookup and the retrieval of the item only requires 2
    	  disk accesses.
      </p>
      <p>
	  When items are removed from the disk cache, the location of the available
	  block on the storage file is recorded in a sorted preferential array of a 
	  size not to exceed the maximum number of keys allowed in memory.  This allows
	  the disk cache to reuse empty spots, thereby keeping the file size to a 
	  minimum.
      </p>
      </subsection>

      <subsection name="Purgatory">
      <p>
	      Writing to the disk cache is asynchronous and made efficient by using a 
        memory staging area called purgatory.  Retrievals check purgatory then
     	  disk for an item.  When items are sent to purgatory they are simultaneously 
        queued to be put to disk.  If an item is retrieved form purgatory it will no
        longer be written to disk, since the cache hub will move it back to memory.  
        Using purgatory insures that there is no wait for disk writes, unecessary 
        disk writes are avoided for boarderline items, and the items are always 
        available.  		
      </p>
      </subsection>

      <subsection name="Persistence">
      <p>
	      When the disk cache is properly shutdown, the memory index is written
        to disk and the value file is defragmented.  When the cache starts
        up, the disk cache can be configured to read or delete the index file.  This 
        provides an unreliable persistence mechanism.   
      </p>
      </subsection>

      <subsection name="Configuration">
        <p>
          The simple configuration and is done in the auxiliary 
          cache section of the <code>cache.ccf</code> configuration file.
          In the example below, I created an Indexed Disk Auxiliary Cache 
          referenced by <code>DC</code>.  It uses files located in the 
          "DiskPath" directory.
         </p>
	   <p>
	      The Disk indexes are equipped with an LRU storage limit.  The maximum 
          number of keys is configured by the maxKeySize parameter.  If the
          maximum key size is less than 0, no limit will be placed on the
          number of keys.  By default, the max key size is 5000.
	   </p>
        <source><![CDATA[
jcs.auxiliary.DC=
    org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
jcs.auxiliary.DC.attributes=
    org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.DC.attributes.DiskPath=g:\dev\jakarta-turbine-stratum\raf
jcs.auxiliary.DC.attributes.MaxKeySize=100000
        ]]></source>
      </subsection>

      <subsection name="Additional Configuration Options">
        <p>
          The indexed disk cache provides some additional configuration options.
	   </p>
	   <p>
	    The purgatory size of the Disk cache is equipped with an LRU storage limit.
	    The maximum number of elements allowed in purgatory is configured by the 
	    MaxPurgatorySize parameter.  By default, the max purgatory size is 5000.
	   </p>
	   <p>
	    Initial testing indicates that the disk cache performs better when the 
	    key and purgatory sizes are limited.
	   </p>
        <source><![CDATA[
jcs.auxiliary.DC.attributes.MaxPurgatorySize=10000
        ]]></source>
	   <p>
	    Slots in the data file become empty when items are removed from the disk cache.
	    The indexed disk cache keeps track of empty slots in the data file, so they can
	    be reused.  The slot locations are stored in a sorted preferential array --
	    the recycle bin.  The smallest items are removed from the recycle bin when it 
	    reaches the specified limit.  The MaxRecycleBinSize cannot be larger than
	    the MaxKeySize.  If the MaxKeySize is less than 0, the recycle bin will default 
	    to 5000.  
	   </p>
        <source><![CDATA[
jcs.auxiliary.DC.attributes.MaxRecycleBinSize=10000
        ]]></source>
	   <p>
	    The Disk cache can be configured to defragment the data file at runtime.  Since
	    defragmentation is only necessary if items have been removed, the deframentation
	    interval is determined by the number of removes.  Currently there is no way
	    to schedule defragmentation to run at a set time.  If you set the OptimizeAtRemoveCount
	    to -1, no optimizations of the data file will occur until shutdown.  By default
	    the value is -1.
	   </p>
        <source><![CDATA[
jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=30000
        ]]></source>
      </subsection>

      <subsection name="A Complete Configuration Example">
        <p>
          In this sample cache.ccf file, I configured the cache to use a disk cache, 
          called DC, by default.  Also, I explicitly set a cache region called myRegion1
          to use DC.  I specified custom settings for all of the Indexed Disk Cache
          configuration parameters. 
	   </p>      
        <source><![CDATA[        
##############################################################
##### Default Region Configuration
jcs.default=DC
jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects=100
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache

##############################################################
##### CACHE REGIONS
jcs.region.myRegion1=DC
jcs.region.myRegion1.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
jcs.region.myRegion1.cacheattributes.MaxObjects=1000
jcs.region.myRegion1.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache

##############################################################
##### AUXILIARY CACHES
# Indexed Disk Cache
jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.DC.attributes.DiskPath=target/test-sandbox/indexed-disk-cache
jcs.auxiliary.DC.attributes.MaxPurgatorySize=10000
jcs.auxiliary.DC.attributes.MaxKeySize=10000
jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=300000
jcs.auxiliary.DC.attributes.MaxRecycleBinSize=7500
        ]]></source>
      </subsection>

       <subsection name="TODO">
        <p>
          The Indexed Disk Auxiliary Cache will eventually be equiped 
          with periodic index storage. 
        </p>
      </subsection>
    </section>
  </body>
</document>