tree f8cf71b123d1e3e59e39d484fb9e8b4858391c55
parent f833ac08c59e576d4cdd80b3ad4c4115bfa6f93d
author Ekta Khanna <ekhanna@pivotal.io> 1611259073 -0800
committer Ekta Khanna <ekhanna@pivotal.io> 1611878648 -0800

DL: Fix gpu mem fraction calc when data isn't distributed to all segs

JIRA: MADLIB-1463

Previously, the calculation of `gpu_mem_fraction` assumed that
num_segments = all_segments which is not always the case. The user can
pass in a distribution rules table to input preprocessor and the data
can be distributed to less segments than the total number of segments on
the cluster.

This commit replaces the get_segments_per_host function call with
get_data_distribution_per_segment which returns the actual distribution
of the data instead of returning a list of all the segments. Using this,
we can calculate the correct memory fraction.

Co-authored-by: Nikhil Kak <nkak@vmware.com>