file cache: evict open fd when descriptor goes out of scope

Previously, an open fd remained in cache when its descriptor was destroyed
unless the file was marked for deletion. This avoided an extra cache lookup
at destruction time and provided faster access to a file if it was closed
and then reopened (though that second benefit was irrelevant to the block
managers where files were kept open permanently unless they were deleted).

With this change, an open fd is forcefully evicted when its descriptor is
destroyed. This provides better semantics if the goal is to close a file
without deleting it and test that the fd is indeed closed. It also prevents
"false positive" cache hits when the user closes the file, renames it, and
creates a new file with the old file name. This is an access pattern used by
the WAL when a replica fails and it is brought back to life via tablet copy
from a healthy server.

Change-Id: Iea5317add630753716ef538cc8a198c9b3547822
Reviewed-on: http://gerrit.cloudera.org:8080/15080
Reviewed-by: Andrew Wong <awong@cloudera.com>
Tested-by: Adar Dembo <adar@cloudera.com>
3 files changed