mini-cluster: disallow restarting daemons from other threads

If we restart an external daemon from a separate thread, the daemon can be
killed silently and without warning when the thread is reaped.

For instance, the following would fail without logging any information about
the tserver dying:

TEST_F(ExternalMiniClusterITestBase, TestRestartFromThread) {
  ExternalMiniClusterOptions opts;
  opts.num_tablet_servers = 1;
  NO_FATALS(StartClusterWithOpts(std::move(opts)));
  thread t([&] {
    auto* ts = cluster_->tablet_server(0);
    ts->Shutdown();
    return ts->Restart();
  });
  t.join();
  SleepFor(MonoDelta::FromSeconds(1));
  ASSERT_TRUE(cluster_->tablet_server(0)->IsProcessAlive());
}

I didn't add a death test, since death tests themselves don't work well in
multithreaded contexts.

Change-Id: I184a01be3e1ac7f60a8b3aedab176dc9138033e0
Reviewed-on: http://gerrit.cloudera.org:8080/15069
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <aserbin@cloudera.com>
2 files changed