tree b286697cc5d45c8d326762416c1b3aaeda19f5df
parent 28cbe037cf9299ed6a089cc78039d0a16d0116ce
author Maja Kabiljo <majakabiljo@fb.com> 1467842253 -0700
committer Maja Kabiljo <majakabiljo@fb.com> 1468258094 -0700

GIRAPH-1085: Add InMemoryDataAccessor

Summary: When we deal with graphs which have a lot of vertices with very little total data associated with them (values + edges) we start experiencing memory problems because of too many objects created, since every vertex has multiple objects associated with it. To solve this problem, we should have a serialized partition representation (current ByteArrayPartition just keeps byte[] per vertex, not per partition). We can leverage the out-of-core infrastructure and just add data accessor which won't be backed by disk but in memory buffers.

Test Plan: Successfully ran a job which was failing without this.

Differential Revision: https://reviews.facebook.net/D60435