大家好,我是考100分的小小码 ,祝大家学习进步,加薪顺利呀。今天说一说Apache HttpClient两种重试机制实现HttpRequestRetryHandler和ServiceUnavailableRetryStrategy,希望您对编程的造诣更进一步.
一、前言
之前遇到过个场景Http请求第三方系统要求重试,看了现有的HttpClient
工具类并没有添加重试。所以对工具类进行改造。
二、编码实现
2.1 前置准备
这里使用Spring Boot 2.5版本
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.10</version>
</parent>
2.2 创建Spring Boot Web应用模拟异常
引入spring web依赖
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
简单创建一个Spring Boot启动类
@SpringBootApplication
public class ThirdServerlication {
public static void main(String[] args) {
SpringApplication.run(ThirdServerlication.class, args);
}
}
创建一个测试用的Controller,接收到请求后线程睡眠10秒,模拟超时。
@RestController
public class TestController {
private static final Logger logger = LoggerFactory.getLogger(TestController.class);
@GetMapping("test/client")
public Object testClient(HttpServletRequest request) {
logger.info("coming in {}", request.getRemoteAddr());
HashMap<Object, Object> map = new HashMap<>();
map.put("error_code", 500);
map.put("msg", "inner server error");
try {
// 线程睡眠10s,模拟超时
TimeUnit.SECONDS.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
return map;
}
}
这里定义的服务使用默认的端口8080,本地测试时请求:http://localhost:8080/test/client
。好了这里模拟第三方服务的应用创建完毕。
2.3 创建并模拟请求第三方接口
同样的创建一个Spring Boot应用,这里端口使用8880
server:
port: 8880
maven依赖:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpasyncclient</artifactId>
</dependency>
Spring Boot启动类:
@SpringBootApplication
public class SimpleApplication {
public static void main(String[] args) {
SpringApplication.run(SimpleApplication.class, args);
}
}
创建HttpClient
工具类:
public class HttpClientUtils {
/** * Timeout (Default is 5s). */
private static final int TIMEOUT = 5000;
private static final Logger logger = LoggerFactory.getLogger(HttpClientUtils.class);
private static final int DEFAULT_TOTAL = 20;
private static final int MAX_TOTAL = 100;
private static PoolingHttpClientConnectionManager poolingHttpClientConnectionManager = null;
static {
poolingHttpClientConnectionManager = new PoolingHttpClientConnectionManager();
poolingHttpClientConnectionManager.setMaxTotal(MAX_TOTAL);
poolingHttpClientConnectionManager.setDefaultMaxPerRoute(DEFAULT_TOTAL);
}
private HttpClientUtils() {
}
/** * Creates http client * author MaiShuRen * * @param timeout connection timeout (ms) * @return http client */
@NonNull
public static CloseableHttpClient createHttpClient(int timeout) {
return resolveProxySetting(HttpClients.custom())
.setDefaultRequestConfig(getRequestConfig(timeout))
.build();
}
/** * Creates http client with retry handler * * @param timeout connection timeout (ms) * @param retryHandler retry handler * @return http client */
@NonNull
public static CloseableHttpClient createHttpClient(int timeout, HttpRequestRetryHandler retryHandler) {
if (Objects.isNull(retryHandler)) {
return createHttpClient(timeout);
}
return resolveProxySetting(HttpClients.custom())
.setDefaultRequestConfig(getRequestConfig(timeout))
.setConnectionManager(poolingHttpClientConnectionManager)
.setRetryHandler(retryHandler)
.build();
}
/** * resolve system proxy config * * @param httpClientBuilder the httpClientBuilder * @return the argument */
private static HttpClientBuilder resolveProxySetting( final HttpClientBuilder httpClientBuilder) {
final String httpProxyEnv = System.getenv("http_proxy");
if (StringUtils.isNotBlank(httpProxyEnv)) {
final String[] httpProxy = resolveHttpProxy(httpProxyEnv);
final HttpHost httpHost = HttpHost.create(httpProxy[0]);
httpClientBuilder.setProxy(httpHost);
if (httpProxy.length == 3) {
//set proxy credentials
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider
.setCredentials(new AuthScope(httpHost.getHostName(), httpHost.getPort()),
new UsernamePasswordCredentials(httpProxy[1], httpProxy[2]));
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
}
}
return httpClientBuilder;
}
/** * Gets request config. * * @param timeout connection timeout (ms) * @return request config */
private static RequestConfig getRequestConfig(int timeout) {
return RequestConfig.custom()
.setConnectTimeout(timeout)
.setConnectionRequestTimeout(timeout)
.setSocketTimeout(timeout)
.build();
}
}
创建测试使用的Controller:
@RestController
public class HttpClientController {
private static final Logger logger = LoggerFactory.getLogger(HttpClientController.class);
private static final int TIMEOUT = 3000;
private static final int RETRY_COUNT = 3;
@GetMapping("test")
public Object test() {
String url = "http://127.0.0.1:8080/test/client";
CloseableHttpClient httpClient = HttpClientUtils.createHttpClient(TIMEOUT, new MyHttpRequestRetryHandler(RETRY_COUNT));
HttpGet httpGet = new HttpGet(url);
logger.info("开始请求: {}", url);
try {
CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
int statusCode = httpResponse.getStatusLine().getStatusCode();
String httpResult = EntityUtils.toString(httpResponse.getEntity());
if (statusCode == HttpStatus.SC_OK) {
logger.info("请求结果:{}", httpResult);
return httpResult;
}
} catch (IOException e) {
logger.error("接口请求异常", e);
}
return "接口异常返回";
}
private static class MyHttpRequestRetryHandler implements HttpRequestRetryHandler {
private final int retryCount;
public MyHttpRequestRetryHandler(int retryCount) {
this.retryCount = retryCount;
}
@Override
public boolean retryRequest(IOException exception, int executionCount, HttpContext httpContext) {
if (executionCount > this.retryCount) {
logger.warn("重试次数已达上限:{}", this.retryCount);
return false;
}
// Unknown host
if (exception instanceof UnknownHostException) {
return false;
}
// SSL handshake exception
if (exception instanceof SSLException) {
return false;
}
if (exception instanceof InterruptedIOException
|| exception instanceof NoHttpResponseException
|| exception instanceof SocketException) {
logger.warn("开始请求重试");
return true;
}
HttpClientContext clientContext = HttpClientContext.adapt(httpContext);
HttpRequest request = clientContext.getRequest();
// Retry if the request is considered idempotent
return !(request instanceof HttpEntityEnclosingRequest);
}
}
}
2.4 运行&测试
分别启动两个Sping Boot应用(SimpleApplication
&ThirdServerlication
),通过浏览器或者PostMan等工具触发
请求:http://localhost:8880/test/
此时查看SimpleApplication
的日志输出:
[nio-8880-exec-1] c.m.b.m.controller.HttpClientController : 开始请求: http://127.0.0.1:8080/test/client
[nio-8880-exec-1] c.m.b.m.controller.HttpClientController : 开始请求重试
[nio-8880-exec-1] o.apache.http.impl.execchain.RetryExec : I/O exception (java.net.SocketTimeoutException) caught when processing request to {}->http://127.0.0.1:8080: Read timed out
[nio-8880-exec-1] o.apache.http.impl.execchain.RetryExec : Retrying request to {}->http://127.0.0.1:8080
[nio-8880-exec-1] c.m.b.m.controller.HttpClientController : 开始请求重试
[nio-8880-exec-1] o.apache.http.impl.execchain.RetryExec : I/O exception (java.net.SocketTimeoutException) caught when processing request to {}->http://127.0.0.1:8080: Read timed out
[nio-8880-exec-1] o.apache.http.impl.execchain.RetryExec : Retrying request to {}->http://127.0.0.1:8080
[nio-8880-exec-1] c.m.b.m.controller.HttpClientController : 开始请求重试
[nio-8880-exec-1] o.apache.http.impl.execchain.RetryExec : I/O exception (java.net.SocketTimeoutException) caught when processing request to {}->http://127.0.0.1:8080: Read timed out
[nio-8880-exec-1] o.apache.http.impl.execchain.RetryExec : Retrying request to {}->http://127.0.0.1:8080
[nio-8880-exec-1] c.m.b.m.controller.HttpClientController : 重试次数已达上限:3
[nio-8880-exec-1] c.m.b.m.controller.HttpClientController : 接口请求异常
java.net.SocketTimeoutException: Read timed out
at java.base/java.net.SocketInputStream.socketRead0(SocketInputStream.java) ~[na:na]
at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115) ~[na:na]
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168) ~[na:na]
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140) ~[na:na]
....
可见超时重试了,达到了我们想要的结果。
三、关于自定义实现HttpRequestRetryHandler的逻辑思考
从上面可见,核心是实现一个HttpRequestRetryHandler
。
HttpRequestRetryHandler
是一个接口用于确定在执行过程中发生异常后是否应该重试,此接口的实现必须是线程安全的,操作共享数据时必须保证同步,因为多个线程发起HTTP请求时使用同一个HttpRequestRetryHandler
的实现并且存在操作共享数据的情况的话会引发线程安全问题。它仅有一个实现方法:
boolean retryRequest(IOException exception, int executionCount, HttpContext context);
对于HttpRequestRetryHandler
接口Apache提供了一个实现类DefaultHttpRequestRetryHandler
,它默认是重试的,但是这个类对于我们研究或者理解重试处理很重要,理解了它的逻辑我们就可以仿写或根据自身的需要来实现重试处理。
先来看一下DefaultHttpRequestRetryHandler
的构造方法:
protected DefaultHttpRequestRetryHandler( final int retryCount, final boolean requestSentRetryEnabled, final Collection<Class<? extends IOException>> clazzes) {
super();
this.retryCount = retryCount;
this.requestSentRetryEnabled = requestSentRetryEnabled;
this.nonRetriableClasses = new HashSet<Class<? extends IOException>>();
for (final Class<? extends IOException> clazz: clazzes) {
this.nonRetriableClasses.add(clazz);
}
}
@SuppressWarnings("unchecked")
public DefaultHttpRequestRetryHandler(final int retryCount, final boolean requestSentRetryEnabled) {
this(retryCount, requestSentRetryEnabled, Arrays.asList(
InterruptedIOException.class,
UnknownHostException.class,
ConnectException.class,
SSLException.class));
}
public DefaultHttpRequestRetryHandler() {
this(3, false);
}
1、不传任何参数时:默认重试三次,不开启自动重试
2、传入重试次数和开启自动重试之后,发生以下的异常不会进行重试:
InterruptedIOException
UnknownHostException
ConnectException
SSLException
在本文编码实现章节那里我们通过线程休眠来模拟第三方服务长时间处理导致的超时,抛出的异常是SocketTimeoutException
,而该异常是继承了InterruptedIOException
,所以如果我们使用了DefaultHttpRequestRetryHandler
作为我们重试处理器,即使开启了自动重试,在发生超时也不会区重试。因为这些超时异常都是继承了InterruptedIOException
,如下图:
所以我们想要自动重试就要自定义实现,而且可以学习一下DefaultHttpRequestRetryHandler
的retryRequest
是怎么写的,在此基础上改造出适合我们自己的HttpRequestRetryHandler
实现。
@Override
public boolean retryRequest( final IOException exception, final int executionCount, final HttpContext context) {
Args.notNull(exception, "Exception parameter");
Args.notNull(context, "HTTP context");
// 超过了最大重试次数之后就不在重试
if (executionCount > this.retryCount) {
return false;
}
// 发生的异常是否是在不重试处理的异常集合中
if (this.nonRetriableClasses.contains(exception.getClass())) {
return false;
}
for (final Class<? extends IOException> rejectException : this.nonRetriableClasses) {
if (rejectException.isInstance(exception)) {
return false;
}
}
final HttpClientContext clientContext = HttpClientContext.adapt(context);
final HttpRequest request = clientContext.getRequest();
if(requestIsAborted(request)){
return false;
}
// 是否考虑幂等处理
if (handleAsIdempotent(request)) {
// Retry if the request is considered idempotent
return true;
}
// 请求未完全发送 || 是否开启重试
if (!clientContext.isRequestSent() || this.requestSentRetryEnabled) {
// Retry if the request has not been sent fully or
// if it's OK to retry methods that have been sent
return true;
}
// 其他情况都不重试
return false;
}
/** * @since 4.2 */
protected boolean handleAsIdempotent(final HttpRequest request) {
return !(request instanceof HttpEntityEnclosingRequest);
}
关于幂等:
从HttpEntityEnclosingRequest
的实现看,**如果是带有是Entity的请求,是不会去重试的。**这是DefaultHttpRequestRetryHandler
的处理,DefaultHttpRequestRetryHandler
有一个子类StandardHttpRequestRetryHandler
它主要是重写了handleAsIdempotent
方法
**StandardHttpRequestRetryHandler:**该类认为所有的HTTP方法实际上都是幂等的。
isRequestSent():
就是获取HTTP请求中的属性http.request_sent
的值是否是false
,如果要用到它的话,可以创建一个它的实现类HttpClientContext
,在调用时传入。例如:
@GetMapping("test")
public Object test() {
String url = "http://127.0.0.1:8080/test/client";
CloseableHttpClient httpClient = HttpClientUtils.createHttpClient(TIMEOUT, new MyHttpRequestRetryHandler(RETRY_COUNT));
HttpClientContext httpClientContext = new HttpClientContext();
httpClientContext.setAttribute("http.request_sent", false);
HttpGet httpGet = new HttpGet(url);
logger.info("开始请求: {}", url);
CloseableHttpResponse httpResponse = null;
try {
httpResponse = httpClient.execute(httpGet, httpClientContext);
int statusCode = httpResponse.getStatusLine().getStatusCode();
String httpResult = EntityUtils.toString(httpResponse.getEntity());
if (statusCode == HttpStatus.SC_OK) {
logger.info("请求结果:{}", httpResult);
return httpResult;
}
} catch (IOException e) {
logger.error("接口请求异常", e);
} finally {
if (httpResponse != null) {
try {
httpResponse.close();
} catch (IOException e) {
logger.error("close response fail ",e);
}
}
}
return "接口异常返回";
}
至此,看到这里对应Apache HttpClient的重试处理HttpRequestRetryHandler
可以说是十分清晰了。
那么,我们再来思考一个问题,我们自己的系统要请求第三方的接口,第三方的文档告诉你,他们会返回例如这样的信息{“code”: 5000,”msg”:”系统繁忙”},如果code的值为5000时,可以尝试重试3次。
毕竟这样也很正常,很多系统都会定义自己的状态码,但是我们通过HttpClient(或其他HTTP客户端)调用收到这样的返回时,HTTP Status是200,第三方系统的业务爪状态码却不是成功的状态码,这样时候他们的文档说可以尝试去重试3次。这就懵了,HTTP Status可是200呀!!是走不到我们的tryHandler
的呀!都没发生异常,而且retryRequest
方法获取不到Response
。这时候应该怎么做呢?这时候估计就各种百度了,可能就是for循环调用了,而且一点都不优雅。
四、重试策略ServiceUnavailableRetryStrategy
Apache HttpClient是一个开源的工具库,设计方面肯定是毋庸置疑的,StandardHttpRequestRetryHandler
是一个重试相关的顶层接口,去这个包下看一下说不定有解决方法。这时候就会发现带有Retry
字眼的类:ServiceUnavailableRetryStrategy
服务不可用时的重试策略。
从StandardHttpRequestRetryHandler
可知,ServiceUnavailableRetryStrategy
应该也是会有一个实现好的一个类。没错它就是有一个实现类:DefaultServiceUnavailableRetryStrategy
。这逻辑可以说时相当简单!!!
@Contract(threading = ThreadingBehavior.IMMUTABLE)
public class DefaultServiceUnavailableRetryStrategy implements ServiceUnavailableRetryStrategy {
/** * Maximum number of allowed retries if the server responds with a HTTP code * in our retry code list. Default value is 1. */
private final int maxRetries;
/** * Retry interval between subsequent requests, in milliseconds. Default * value is 1 second. */
private final long retryInterval;
public DefaultServiceUnavailableRetryStrategy(final int maxRetries, final int retryInterval) {
super();
Args.positive(maxRetries, "Max retries");
Args.positive(retryInterval, "Retry interval");
this.maxRetries = maxRetries;
this.retryInterval = retryInterval;
}
public DefaultServiceUnavailableRetryStrategy() {
this(1, 1000);
}
@Override
public boolean retryRequest(final HttpResponse response, final int executionCount, final HttpContext context) {
// 重试次数小于等于最大重试次数
return executionCount <= maxRetries &&
// Http Status等于503
response.getStatusLine().getStatusCode() == HttpStatus.SC_SERVICE_UNAVAILABLE;
}
@Override
public long getRetryInterval() {
return retryInterval;
}
}
两个属性:最大重试次数和重试时间间隔。默认重试三次、每隔一秒重试。
4.1 自定义实现ServiceUnavailableRetryStrategy
这里引入gson库
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
</dependency>
编写重试策略
public class MyServiceUnavailableRetryStrategy implements ServiceUnavailableRetryStrategy {
private static final Logger logger = LoggerFactory.getLogger(MyServiceUnavailableRetryStrategy.class);
private final int maxRetries;
private final long retryInterval;
public MyServiceUnavailableRetryStrategy(final int maxRetries, final long retryInterval) {
this.maxRetries = maxRetries;
this.retryInterval = retryInterval;
}
public MyServiceUnavailableRetryStrategy() {
this(3, 1000);
}
@Override
public boolean retryRequest(HttpResponse response, int executionCount, HttpContext context) {
logger.info("Come in MyServiceUnavailableRetryStrategy");
if (executionCount > maxRetries) {
logger.warn("RetryStrategy 重试次数已达:{}", maxRetries);
return false;
}
if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
try {
String resultResp = EntityUtils.toString(response.getEntity());
JsonObject jsonObject = JsonParser.parseString(resultResp).getAsJsonObject();
// 假设第三方请求返回业务状态码使用error_code字段
int code = jsonObject.get("error_code").getAsInt();
if (code == 5000) {
logger.info("开始第{}重试", executionCount);
return true;
}
} catch (IOException e) {
logger.error("读取结果异常");
}
}
// Http Status 503 也重试
return response.getStatusLine().getStatusCode() == HttpStatus.SC_SERVICE_UNAVAILABLE;
}
@Override
public long getRetryInterval() {
return this.retryInterval;
}
}
Apache HttpClient默认是没有设置ServiceUnavailableRetryStrategy
,所以在创建HTTP Client设置ServiceUnavailableRetryStrategy
。在HttpClientUtils
里面添加重载方法
@NonNull
public static CloseableHttpClient createHttpsClient(int timeout, HttpRequestRetryHandler retryHandler, ServiceUnavailableRetryStrategy retryStrategy)
throws KeyStoreException, NoSuchAlgorithmException, KeyManagementException {
if (Objects.isNull(retryHandler)) {
return createHttpsClient(timeout);
}
if (Objects.isNull(retryStrategy)) {
return createHttpsClient(timeout, retryHandler);
}
SSLContext sslContext = new SSLContextBuilder()
.loadTrustMaterial(null, (certificate, authType) -> true)
.build();
return resolveProxySetting(HttpClients.custom())
.setSSLContext(sslContext)
.setSSLHostnameVerifier(new NoopHostnameVerifier())
.setRetryHandler(retryHandler)
.setDefaultRequestConfig(getRequestConfig(timeout))
.build();
}
@NonNull
public static CloseableHttpClient createHttpClient(int timeout, HttpRequestRetryHandler retryHandler, ServiceUnavailableRetryStrategy retryStrategy) {
if (Objects.isNull(retryHandler)) {
return createHttpClient(timeout);
}
if (Objects.isNull(retryStrategy)) {
return createHttpClient(timeout, retryHandler);
}
return resolveProxySetting(HttpClients.custom())
.setDefaultRequestConfig(getRequestConfig(timeout))
.setConnectionManager(poolingHttpClientConnectionManager)
.setRetryHandler(retryHandler)
.setServiceUnavailableRetryStrategy(retryStrategy)
.build();
}
在获取HttpClient时使用上面的方法获取
CloseableHttpClient httpClient = HttpClientUtils.createHttpClient(TIMEOUT, new MyHttpRequestRetryHandler(RETRY_COUNT),
new MyServiceUnavailableRetryStrategy());
调整TestController
里的测试代码
@GetMapping("test/client")
public Object testClient(HttpServletRequest request) {
logger.info("coming in {}", request.getRemoteAddr());
HashMap<Object, Object> map = new HashMap<>();
map.put("error_code", 5000);
map.put("msg", "业务繁忙请重试");
// 可同时打开下面线程休眠的注解,观察一下日志打印情况
//try {
// TimeUnit.SECONDS.sleep(10);
//} catch (InterruptedException e) {
// e.printStackTrace();
//}
return map;
}
4.2 运行&测试
分别启动两个Sping Boot应用(SimpleApplication
&ThirdServerlication
),通过浏览器或者PostMan等工具触发
请求:http://localhost:8880/test/
此时查看SimpleApplication
的日志输出:
INFO 10604 --- [nio-8880-exec-4] c.m.b.m.controller.HttpClientController : 开始请求: http://127.0.0.1:8080/test/client
INFO 10604 --- [nio-8880-exec-4] .b.m.h.MyServiceUnavailableRetryStrategy : Come in MyServiceUnavailableRetryStrategy
INFO 10604 --- [nio-8880-exec-4] .b.m.h.MyServiceUnavailableRetryStrategy : 开始第1重试
INFO 10604 --- [nio-8880-exec-4] .b.m.h.MyServiceUnavailableRetryStrategy : Come in MyServiceUnavailableRetryStrategy
INFO 10604 --- [nio-8880-exec-4] .b.m.h.MyServiceUnavailableRetryStrategy : 开始第2重试
INFO 10604 --- [nio-8880-exec-4] .b.m.h.MyServiceUnavailableRetryStrategy : Come in MyServiceUnavailableRetryStrategy
INFO 10604 --- [nio-8880-exec-4] .b.m.h.MyServiceUnavailableRetryStrategy : 开始第3重试
INFO 10604 --- [nio-8880-exec-4] .b.m.h.MyServiceUnavailableRetryStrategy : Come in MyServiceUnavailableRetryStrategy
WARN 10604 --- [nio-8880-exec-4] .b.m.h.MyServiceUnavailableRetryStrategy : RetryStrategy 重试次数已达:3
INFO 10604 --- [nio-8880-exec-4] c.m.b.m.controller.HttpClientController : 请求结果:{"msg":"业务繁忙请重试","error_code":5000}
哈哈!Nice!
五、总结
这里用一张图总结,可以说HttpRequestRetryHandler
是在请求的时候发生异常的话,可根据自身需要去判断发生的异常是否需要重试。ServiceUnavailableRetryStrategy
则是在拿到第三方的请求结果的时候,去判断的。
版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 举报,一经查实,本站将立刻删除。
转载请注明出处: https://daima100.com/13916.html