tree b0f24055cdf6bf2deb27c380be5de0f07b701014
parent 1c656aad3495814fb55b5c2f5be708642b6174f3
author runzhiwang <runzhiwang@tencent.com> 1576736941 +0800
committer jerryshao <jerryshao@tencent.com> 1576736941 +0800

[LIVY-727] Fix session state always be idle though the yarn application has been killed after restart livy

## What changes were proposed in this pull request?

[LIVY-727] Fix session state always be idle though the yarn application has been killed after restart livy.

Follows are steps to reproduce the problem:
1. Set livy.server.recovery.mode=recovery, and create a session in yarn-cluster
2. Restart livy
3. Kill the yarn application of the session.
4. The session state will always be idle and never change to killed or dead. Just as the image, livy-session-16 has been killed in yarn, but the state is still idle.
![image](https://user-images.githubusercontent.com/51938049/70371088-92695c80-1909-11ea-875c-73696db693ce.png)

The cause of the problem are as follows:
1. Because when recover session, livy will not startDriver again, so the driverProcess is None.
2. SparkYarnApp will not be created in `driverProcess.map { _ => SparkApp.create(appTag, appId, driverProcess, livyConf, Some(this)) }` when driverProcess is None.
3. So yarnAppMonitorThread of the session will never start, and the session state will never change.

How to fix the bug:
1. If livy run in yarn, SparkApp will create even though the driverProcess is None
2. If not run in yarn, SparkApp will not create, because the code require driverProcess is not None at https://github.com/apache/incubator-livy/blob/master/server/src/main/scala/org/apache/livy/utils/SparkApp.scala#L93,
  and I don't want to change the behavior.

## How was this patch tested?

1. Set livy.server.recovery.mode=recovery, and create a session in yarn-cluster
2. Restart livy
3. Kill the yarn application of the session.
4. The session state will change to killed

Author: runzhiwang <runzhiwang@tencent.com>

Closes #265 from runzhiwang/session-state.