Make manifest cache size configurable (#2993)
<!--
Thanks for opening a pull request!
-->
<!-- In the case this PR will resolve an issue, please replace
${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${2952} -->
Closes #2952
# Rationale for this change
Through discussion in issue #2325, we realized that there was a memory
leak in the manifest cache. PR
https://github.com/apache/iceberg-python/pull/2951 fixed this memory
leak, but we decided that it would be best for developer experience if
users could configure the cache for their needs.
This includes the ability to configure the manifest cache size,
including setting it to `0` to disable caching.
## Are these changes tested?
Yes - unit tests are included to verify these changes
## Are there any user-facing changes?
Yes:
- `manifest-cache-size` can be used in `.pyiceberg.yaml` to configure
the size of the manifest cache
- `PYICEBERG_MANIFEST_CACHE_SIZE` can be used to configure the size of
the manifest cache
<!-- In the case of user-facing changes, please add the changelog label.
-->
---------
Co-authored-by: Kevin Liu <kevin.jq.liu@gmail.com>PyIceberg is a Python library for programmatic access to Iceberg table metadata as well as to table data in Iceberg format. It is a Python implementation of the Iceberg table spec.
The documentation is available at https://py.iceberg.apache.org/.