Jsoup - get number of redirects (http 301 or 302)
In this article I would like to show how to get the number of redirects with Jsoup Java library.
I didn't find any easy way to get this number, but I needed to make it done for unit tests. I need to have max 1 redirect to final URL. It can be accomplished by using reflection and making internal priavte Jsoup variable public. The variable name is: numRedirects and it is not accessible from Jsoup API level.
I found it by digging into the Jsoup internals, because I couldn't find any answer on the internet.
On below example the number of redirects is 1 and status code is 302 (HTTP response status code 302 Found).
Quick solution:
private static int getRedirectsNumber(org.jsoup.Connection.Response response)
throws NoSuchFieldException, IllegalAccessException {
// we use reflection to make numRedirects public
Field field = response.getClass().getDeclaredField("numRedirects");
field.setAccessible(true);
return (int) field.get(response);
}
Practical example
Full working code example:
import org.jsoup.Connection;
import org.jsoup.Jsoup;
import java.lang.reflect.Field;
import java.net.URL;
public class JsoupRedirections {
public static void main(String[] args) throws Exception {
String url = "https://dirask.com/posts/1X9GgD";
Connection connection = Jsoup.connect(url)
.followRedirects(true)
.userAgent("Mozilla/5.0 AppleWebKit/537.36 (KHTML, " +
"like Gecko) Chrome/45.0.2454.4 Safari/537.36")
.method(Connection.Method.GET)
.timeout(10_000); // 10 sec
Connection.Response response = connection.execute();
int redirectsNumber = getRedirectsNumber(response);
System.out.println("redirects number: " + redirectsNumber); // 1
URL resultUrl = response.url();
// https://dirask.com/posts/Jsoup-get-number-of-redirects-http-301-or-302-1X9GgD
System.out.println(resultUrl);
}
private static int getRedirectsNumber(Connection.Response response)
throws NoSuchFieldException, IllegalAccessException {
// we use reflection to make numRedirects public
Field field = response.getClass().getDeclaredField("numRedirects");
field.setAccessible(true);
return (int) field.get(response);
}
}
Output:
redirects number: 1
https://dirask.com/posts/Jsoup-get-number-of-redirects-http-301-or-302-1X9GgD
Maven dependency for Jsoup version 1.11.3: (attach it to *.pom file):
<!-- https://mvnrepository.com/artifact/org.jsoup/jsoup -->
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.11.3</version>
</dependency>
Explaination
This section shows where number of redirects is placed inside Jsoup library. Above solution extracted the value with reflection.
The Jsoup Response internals:
public class HttpConnection implements Connection {
// ...
public static class Response extends HttpConnection.Base<Connection.Response> implements Connection.Response {
private static final int MAX_REDIRECTS = 20;
private static SSLSocketFactory sslSocketFactory;
private static final String LOCATION = "Location";
private int statusCode;
private String statusMessage;
private ByteBuffer byteData;
private InputStream bodyStream;
private String charset;
private String contentType;
private boolean executed = false;
private boolean inputStreamRead = false;
private int numRedirects = 0; // <-------------------- num redirects
private Connection.Request req;
// ...
Screenshot: