tree bc02051ad0c05a037bd075c2b85bc6f1efd8ce23
parent 110b99d1c88f3d34e9edbe40d957609421d35c25
author Matteo Merli <mmerli@apache.org> 1607760137 -0800
committer GitHub <noreply@github.com> 1607760137 +0800
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJf1HkJCRBK7hj4Ov3rIwAAdHIIAG2DcoNJ+ncMI1NJ0xvuq2n4
 N/pQlU/SaxnBPuVAXrTDPkxTEklyNf44EHKpL5aDrA2/8Iqvw7Kh+su6CPmjXIKZ
 htZ23ylgeC26tTG0rRpvU2iyGeySoO2yoOdXTxPFS5TnciMIqdiCZlX7iFmGiAGH
 /KEQ5RFOFU6E15rYunaGBj8W58Y8ZrSsqavX+nnnWPFh39J+A0Wqj++w1RjN4LA/
 zo3Ssx0OmnZKBYC7U7WT0Zqr44Z/r5gil9mSVihxMz7MP9NVxudYEyALG0NR1Dms
 Bp6ss50+I7poeY1cOLRlIrR0abehoa7nhmocp0u3FMjoOXa9kGBbxrWby+ZEqXA=
 =Io/M
 -----END PGP SIGNATURE-----
 

Fixed logic to attempt reconnections to same broker (#414)

### Motivation

There is a problem with the re-connection logic introduced in #157. 

The change added a logic to keep retrying to establish a TCP connection with broker up to the "operation timeout" (default 30seconds). 

There are few issues with it: 
 1. (minor) It's not checking that the error is indeed a TCP error (eg: it would retry on auth failures too)
 2. (major) After a TCP connection failure, reconnecting to the same broker is always the wrong approach, because the most likely outcome is that the next attempt will also fail and, worse, the IP might just be unresponsive and we will then have to wait for the full connection timeout time. 

The correct solution after a connection failure is to re-do the topic lookup, since the topic will be moving to a different broker and we need to reconnect to the new broker asap. 

The only time we can do this connection retry logic is for requests that are not specific to a particular broker (eg: lookup operations). In this case a quick retry on a connection failure will probably land the request on a different, healthy, broker.