hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt

  BREAKDOWN OF HDFS-6581 SUBTASKS AND RELATED JIRAS

    HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal)

    HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal)

    HDFS-6922. Add LazyPersist flag to INodeFile, save it in FsImage and
    edit logs. (Arpit Agarwal)
  
    HDFS-6923. Propagate LazyPersist flag to DNs via DataTransferProtocol.
    (Arpit Agarwal)

    HDFS-6925. DataNode should attempt to place replicas on transient storage
    first if lazyPersist flag is received. (Arpit Agarwal)

    HDFS-6926. DN support for saving replicas to persistent storage and
    evicting in-memory replicas. (Arpit Agarwal)

    HDFS-6927. Initial unit tests for lazy persist files. (Arpit Agarwal)

    HDFS-6929. NN periodically unlinks lazy persist files with missing
    replicas from namespace. (Arpit Agarwal)

    HDFS-6928. 'hdfs put' command should accept lazyPersist flag for testing.
    (Arpit Agarwal)

    HDFS-6960. Bugfix in LazyWriter, fix test case and some refactoring.
    (Arpit Agarwal)

    HDFS-6931. Move lazily persisted replicas to finalized directory on DN
    startup. (Arpit Agarwal)

    HDFS-6950. Add Additional unit tests for HDFS-6581. (Xiaoyu Yao via
    Arpit Agarwal)

    HDFS-6930. Improve replica eviction from RAM disk. (Arpit Agarwal)

    HDFS-6977. Delete all copies when a block is deleted from the block space.
    (Arpit Agarwal)

    HDFS-6991. Notify NN of evicted block before deleting it from RAM disk.
    (Arpit Agarwal)

    HDFS-6978. Directory scanner should correctly reconcile blocks on RAM
    disk. (Arpit Agarwal)

    HDFS-7066. LazyWriter#evictBlocks misses a null check for replicaState.
    (Xiaoyu Yao via Arpit Agarwal)

    HDFS-7064. Fix unit test failures in HDFS-6581 branch. (Xiaoyu Yao via
    Arpit Agarwal)

    HDFS-6581. Few more unit test fixes for HDFS-6581. (Arpit Agarwal)

    HDFS-7080. Fix finalize and upgrade unit test failures. (Arpit Agarwal)

    HDFS-7084. FsDatasetImpl#copyBlockFiles debug log can be improved.
    (Xiaoyu Yao via Arpit Agarwal)

    HDFS-7091. Add forwarding constructor for INodeFile for existing callers.
    (Arpit Agarwal)

    HDFS-7100. Make eviction scheme pluggable. (Arpit Agarwal)

    HDFS-7108. Fix unit test failures in SimulatedFsDataset. (Arpit Agarwal)

    HDFS-7071. Updated editsStored and editsStored.xml to bump layout
    version and add LazyPersist flag. (Xiaoyu Yao and Arpit Agarwal via
    Arpit Agarwal)
    
    HDFS-6990. Add unit test for evict/delete RAM_DISK block with open
    handle. (Xiaoyu Yao via Arpit Agarwal)

    HDFS-7143. Fix findbugs warnings in HDFS-6581 branch. (szetszwo via
    Arpit Agarwal)

    HDFS-6932. Balancer and Mover tools should ignore replicas on RAM_DISK.
    (Xiaoyu Yao via Arpit Agarwal)

    HDFS-7144. Fix findbugs warnings in RamDiskReplicaTracker. (szetszwo via
    Arpit Agarwal)

    HDFS-7155. Bugfix in createLocatedFileStatus caused by bad merge.
    (Arpit Agarwal)

    HDFS-7153. Add storagePolicy to NN edit log during file creation.
    (Arpit Agarwal)

    HDFS-7159. Use block storage policy to set lazy persist preference.
    (Arpit Agarwal)

    HDFS-7129. Metrics to track usage of memory for writes. (Xiaoyu Yao
    via Arpit Agarwal)

    HDFS-7171. Fix Jenkins failures in HDFS-6581 branch. (Arpit Agarwal)