Hi guys,
Request you to check out the previous post before proceeding – https://blog.niteshapte.com/2018-03-22-how-to-create-a-generic-executor-service-for-asynchronous-calls.htm. This is basically to give the idea how we are going to achieve multiple URL calls – asynchronously.
Below solution is good at those places where we need to get data from different rest services in parallel. Using the received data will obviously depend on your requirement.
The idea is to use a thread pool that can process in bunch.
For the sake of example, we are going to use Fake Online REST API for Testing and Prototyping provided by JSONPlaceHolder – https://jsonplaceholder.typicode.com/
Let’s start with the code now.
Let’s first create a REST client to fetch data. I have used glassfish jersey api. You can use whatever you like.
JerseyRestClient
package com.executor.service;
import java.net.SocketTimeoutException;
import javax.ws.rs.ProcessingException;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import org.glassfish.jersey.client.ClientProperties;
/**
* The Class JerseyRestClient.
*/
public final class JerseyRestClient {
/** The instance. */
private static JerseyRestClient instance;
/** The client. */
private Client client;
/**
* Instantiates a new jersey rest client.
*/
private JerseyRestClient() {
if(client == null) {
createClient();
}
}
/**
* Creates the client.
*/
public void createClient() {
client = ClientBuilder.newClient();
client.property(ClientProperties.CONNECT_TIMEOUT, 3000);
client.property(ClientProperties.READ_TIMEOUT, 3000);
}
/**
* Gets the single instance of JerseyRestClient.
*
* @return single instance of JerseyRestClient
*/
public static synchronized JerseyRestClient getInstance() {
return instance == null ? new JerseyRestClient() : instance;
}
/**
* Gets the response as string.
*
* @param resourceLocation the resource location
* @return the response as string
*/
public String getResponseAsString(String resourceLocation) {
String output = null;
try {
WebTarget target = client.target(resourceLocation);
int i = 0;
int retryNumber = 2;
Response res = null;
while (true) {
try {
res = target.request().accept(MediaType.APPLICATION_JSON).get();
break;
} catch (ProcessingException e){
if (e.getCause() instanceof SocketTimeoutException && i < retryNumber) {
i++;
} else {
break;
}
}
}
if(res.getStatus() == 200) {
output = res.readEntity(String.class);
}
res.close();
} catch (Exception e) {
e.printStackTrace();
}
return output;
}
}
Since, I am using maven here, the pom.xml file will look like this –
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.executor</groupId>
<artifactId>service</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>2.23.1</version>
</dependency>
</dependencies>
</project>
Glassfish jersey api is not a part of Java, so you will need to add it as dependency in your pom.xml file.
Now comes the asynchronous implementation for calling multiple URLs.
AsyncExecutor
package com.executor.service;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.TimeUnit;
/**
* The Class AsyncExecutor.
*/
public class AsyncExecutor {
/** The client. */
JerseyRestClient client = JerseyRestClient.getInstance();
/** The Constant urls. */
private final static String[] urls = {"https://jsonplaceholder.typicode.com/posts", "https://jsonplaceholder.typicode.com/comments", "https://jsonplaceholder.typicode.com/albums", "https://jsonplaceholder.typicode.com/photos", "https://jsonplaceholder.typicode.com/todos", "https://jsonplaceholder.typicode.com/users"};
/**
* Make async calls.
*/
public void makeAsyncCalls() {
List<String> listOfURLs = Arrays.asList(urls);
ExecutorServiceProvider<Void> executorServiceProvider = new ExecutorServiceProvider<Void>();
ExecutorCompletionService<Void> executorCompletionService = new ExecutorCompletionService<Void>(executorServiceProvider.getExecutorService(5));
listOfURLs.stream().filter(f -> f != null && !f.equals("")).forEach(url -> {
Callable<Void> worker = () -> {
callIndividualURLs(url);
return null;
};
try {
executorCompletionService.submit(worker);
} catch (Exception e) {
e.printStackTrace();
}
});
executorServiceProvider.getResult(listOfURLs.size(), executorCompletionService, 5, TimeUnit.SECONDS);
executorServiceProvider.shutdownExecutorService();
}
/**
* Call individual UR ls.
*
* @param url the url
* @return the string
*/
public String callIndividualURLs(String url) {
System.out.println(url);
String data = client.getResponseAsString(url);
System.out.println(data);
return data;
}
}
You must be wondering what and where is ExecutorServiceProvider class. If you are, then you have not read that first statement of this blog :D. But anyway, I will still make it easy for you. You can check it out here – https://github.com/niteshapte/generic-executor-service-java
Only one thing left to do – execute it.
Main
package com.executor.service;
public class Main {
public static void main(String[] args) {
AsyncExecutor asyncExecutor = new AsyncExecutor();
asyncExecutor.makeAsyncCalls();
}
}
Execute it. Have fun.
And that’s it.
You can browse the complete code from GitHub: https://github.com/niteshapte/multiple-url-asynchronous-call-java8
and download directly: https://github.com/niteshapte/multiple-url-asynchronous-call-java8/archive/master.zip
Feedback / comments are welcome.
Have a nice day ahead.
Leave a Reply