Remove chardet from images (#91)
* CM build w/o chardet
* GI build w/o chardet
* Reworked collection manager dependencies
* Update CHANGELOG.md
* GI: Fallback to conda if mamba is not present
---------
Co-authored-by: rileykk <rileykk@jpl.nasa.gov>
diff --git a/CHANGELOG.md b/CHANGELOG.md
index fc71a6c..929e6c4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,13 +4,15 @@
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-## [1.2.0] - 2023-09-14
+## [1.2.0] - 2023-11-22
### Added
- SDAP-477: Added preprocessor to properly shape incoming data
- SDAP-478: Add support to user to select subgroup of interest in input granules
### Changed
+- Changed granule ingester setup to use mamba instead of conda
### Deprecated
### Removed
+- SDAP-501: Updated dependencies to remove `chardet`
### Fixed
- SDAP-488: Workaround to build issue on Apple Silicon (M1/M2). GI image build installs nexusproto through PyPI instead of building from source. A build arg `BUILD_NEXUSPROTO` was defined to allow building from source if desired/
### Security
diff --git a/collection_manager/docker/Dockerfile b/collection_manager/docker/Dockerfile
index f729fa2..5c48215 100644
--- a/collection_manager/docker/Dockerfile
+++ b/collection_manager/docker/Dockerfile
@@ -29,8 +29,6 @@
COPY collection_manager/docker/entrypoint.sh /entrypoint.sh
RUN cd /common && python setup.py install
-RUN cd /collection_manager && python setup.py install
-
-RUN pip install boto3==1.16.10
+RUN cd /collection_manager && python setup.py install && pip uninstall -y chardet && pip install boto3==1.16.10 requests==2.26.0
ENTRYPOINT ["/bin/bash", "/entrypoint.sh"]
diff --git a/collection_manager/requirements.txt b/collection_manager/requirements.txt
index fe82ccf..fbace6e 100644
--- a/collection_manager/requirements.txt
+++ b/collection_manager/requirements.txt
@@ -17,8 +17,10 @@
pystache==0.5.4
pysolr==3.9.0
watchdog==0.10.2
-requests==2.23.0
+requests==2.31.0
tenacity==6.2.0
-aioboto3==8.0.5
-aiohttp==3.7.2
-aio-pika==6.7.1
\ No newline at end of file
+aioboto3==8.3.0
+aiobotocore==1.2.2
+aiohttp==3.8.3
+aio-pika==6.7.1
+urllib3==1.25.8
diff --git a/granule_ingester/conda-requirements.txt b/granule_ingester/conda-requirements.txt
index e49a9eb..f013d3b 100644
--- a/granule_ingester/conda-requirements.txt
+++ b/granule_ingester/conda-requirements.txt
@@ -20,7 +20,7 @@
pytz==2019.3
xarray>=0.19.0
pyyaml==5.3.1
-aiohttp==3.6.2
+aiohttp>=3.8.0
tenacity
-requests==2.27.1
+requests>=2.27.1
diff --git a/granule_ingester/docker/Dockerfile b/granule_ingester/docker/Dockerfile
index e285872..d01c74c 100644
--- a/granule_ingester/docker/Dockerfile
+++ b/granule_ingester/docker/Dockerfile
@@ -35,8 +35,13 @@
ARG APACHE_NEXUSPROTO_BRANCH=master
RUN ./install_nexusproto.sh
-RUN cd /common && python setup.py install
-RUN cd /sdap && python setup.py install
+RUN cd /common && python setup.py install clean
+
+RUN pip uninstall -y chardet
+
+RUN conda install -n base --override-channels -c conda-forge -c defaults mamba libarchive=3.5.2=hb890918_3 && conda clean -afy
+
+RUN cd /sdap && python setup.py install clean && mamba clean -afy
RUN pip install boto3==1.16.10
diff --git a/granule_ingester/setup.py b/granule_ingester/setup.py
index e99900d..8967b22 100644
--- a/granule_ingester/setup.py
+++ b/granule_ingester/setup.py
@@ -24,9 +24,13 @@
__version__ = f.readline()
try:
- check_call(['conda', 'install', '-y', '-c', 'conda-forge', '--file', 'conda-requirements.txt'])
+ check_call(['mamba', 'install', '-y', '-c', 'conda-forge', '--file', 'conda-requirements.txt'])
except (CalledProcessError, IOError) as e:
- raise EnvironmentError("Error installing conda packages", e)
+ print('Failed install with mamba; falling back to conda')
+ try:
+ check_call(['conda', 'install', '-y', '-c', 'conda-forge', '--file', 'conda-requirements.txt'])
+ except (CalledProcessError, IOError) as e:
+ raise EnvironmentError("Error installing conda packages", e)
setup(
name='sdap_granule_ingester',
