28cbe037cf9299ed6a089cc78039d0a16d0116ce - giraph

commit	28cbe037cf9299ed6a089cc78039d0a16d0116ce	[log] [tgz]
author	Maja Kabiljo <majakabiljo@fb.com>	Fri Jul 01 07:39:25 2016 -0700
committer	Maja Kabiljo <majakabiljo@fb.com>	Thu Jul 07 15:45:33 2016 -0700
tree	b28245777117c5abc07e5aece3cd341187786699
parent	819f293f4c780fc6833785da27e10f965570f44e [diff]

GIRAPH-1082: Remove limit on the number of partitions

Summary: Currently we have a limit on how many partitions we can have because we write all partition information to Zookeeper. We can instead send this information in requests and remove the hard limit.

Test Plan: Ran pagerank for 100 iterations with 500k partitions.

Differential Revision: https://reviews.facebook.net/D60267

giraph-core/src/main/java/org/apache/giraph/bsp/BspService.java[diff]
giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java[diff]
giraph-core/src/main/java/org/apache/giraph/comm/requests/AddressesAndPartitionsRequest.java[Added - diff]
giraph-core/src/main/java/org/apache/giraph/comm/requests/PartitionStatsRequest.java[Added - diff]
giraph-core/src/main/java/org/apache/giraph/comm/requests/RequestType.java[diff]
giraph-core/src/main/java/org/apache/giraph/graph/AddressesAndPartitionsWritable.java[diff]
giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java[diff]
giraph-core/src/main/java/org/apache/giraph/master/MasterGlobalCommHandler.java[diff]
giraph-core/src/main/java/org/apache/giraph/partition/PartitionUtils.java[diff]
giraph-core/src/main/java/org/apache/giraph/utils/BlockingElementsSet.java[Added - diff]
giraph-core/src/main/java/org/apache/giraph/utils/ProgressableUtils.java[diff]
giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java[diff]

12 files changed