tree 8c03ee08983b4569fe09a75ffde2353ab844d20f
parent 9b9264523c58a958eabde3b41f778374bd473fc2
author QiangCai <qiangcai@qq.com> 1598256954 +0800
committer akashrn5 <akashnilugal@gmail.com> 1598371146 +0530

[CARBONDATA-3958] Avoid blocking the loading task when the output queue poll timeout

Why is this PR needed?
In some cases, the CDC merge tasks are blocked when data loading uses
CarbonTableOutputFormat.getRecordWriter method. PR #3856 change the code to avoid using
CarbonTableOutputFormat.getRecordWriter, CDC merge will not happen this issue again.
But this issue maybe happen in other similar scenarios.

Because the poll method of the queue is time out in some cases, so the row batch in
the queue will not be polled again.

After the queue is full, the queue blocks the writing task puts a new batch. Even if
the queue is not full, it will also lose the row batch in the queue.

What changes were proposed in this PR?
If the output is not closed, it will poll a row batch in the loops till it gets a not
null batch.
If the output is closed, it will break the loop.

Does this PR introduce any user interface change?
No

Is any new testcase added?
No

This closes #3897