A network call is a request made to a server to retrieve or send data. When a user interacts with a website, various network calls are made in the background to retrieve information from the server. For example, when a user clicks on a button, a network call is made to retrieve the data associated with that button. So, it is important to know what requests were made and the responses that we got for the same.
We already discussed getting the network call requests using Selenium, and this post will focus on getting the network call responses.
We can capture the network calls using the browser developer tools. Most modern web browsers have built-in developer tools that allow us to view and capture network calls.
We can access the developer tools by right-clicking on the webpage and selecting the “inspect” option.
Clicking on the “Inspect” option will open the developer tools, where there are a bunch of tabs. We can click the “Network” tab to capture the network calls.
Capturing network call responses using Selenium
Now, let’s see how we can get the network call responses using selenium.
Selenium 4 has introduced a new feature called DevTools, which allows us to interact with the browser developer tools. We will also use the same to get the network call responses.
Note: In our code, you’ll see an import “org.openqa.selenium.devtools.v108.network.Network” showing a version of 108. This is because we are currently using chrome version 109, and v108 import is compatible with it. Please use the version that is compatible with your chrome browser.
Create a session with DevTools
DevTools devTools = ((ChromeDriver) driver).getDevTools();devTools.createSession();
Enable Network tracking
devTools.send(Network.enable(Optional.of(1000000), Optional.empty(), Optional.empty()));
Network.enable method enables the network tracking. It accepts three parameters – maxTotalBufferSize, maxResourceBufferSize and maxPostDataSize.
All the parameters are to be sent in Bytes.
Capturing response information of a network call
We can easily capture the response of network calls using the below code –
devTools.addListener(Network.responseReceived(), responseReceived -> {
}
Network.responseReceived is fired whenever an HTTP response is received, and the value will be stored in the responseReceived variable. We can then use the stored value to get the network call’s response headers, response status, or response body.
responseReceived.getResponse() will return a Response object which provides many helpful methods to get information about an HTTP response.
How to get the response headers of a Network call using Selenium?
responseReceived.getResponse().getHeaders() gets the headers of a response.
devTools.addListener(Network.responseReceived(), responseReceived -> {
System.out.println("Response headers: " + responseReceived.getResponse().getHeaders().toString());
});
Whole Code
import java.util.Optional;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v108.network.Network;
public class CodekruTest {
public static void main(String[] args) {
// pass the path of the chromedriver location in the second argument
System.setProperty("webdriver.chrome.driver", "E:\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
DevTools devTools = ((ChromeDriver) driver).getDevTools();
devTools.createSession();
devTools.send(Network.enable(Optional.of(1000000), Optional.empty(), Optional.empty()));
devTools.addListener(Network.responseReceived(), responseReceived -> {
String responseUrl = responseReceived.getResponse().getUrl();
if (responseUrl.contains("makemytrip")) {
System.out.println("Url: " + responseUrl);
System.out.println("Response headers: " + responseReceived.getResponse().getHeaders().toString());
}
});
driver.get("https://www.makemytrip.com");
}
}
How to get the response status of a Network call using Selenium?
responseReceived.getResponse().getStatus() gets the response status of an HTTP response.
devTools.addListener(Network.responseReceived(), responseReceived -> {
System.out.println("Response status: " + responseReceived.getResponse().getStatus());
});
Whole Code
import java.util.Optional;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v108.network.Network;
public class CodekruTest {
public static void main(String[] args) {
// pass the path of the chromedriver location in the second argument
System.setProperty("webdriver.chrome.driver", "E:\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
DevTools devTools = ((ChromeDriver) driver).getDevTools();
devTools.createSession();
devTools.send(Network.enable(Optional.of(1000000), Optional.empty(), Optional.empty()));
devTools.addListener(Network.responseReceived(), responseReceived -> {
String responseUrl = responseReceived.getResponse().getUrl();
if (responseUrl.contains("makemytrip")) {
System.out.println("Url: " + responseUrl);
System.out.println("Response status: " + responseReceived.getResponse().getStatus());
}
});
driver.get("https://www.makemytrip.com");
}
}
How to get the response body of a Network call using Selenium?
It is a two-step process –
- First, get the request id of a network call using
responseReceived.getRequestId()method. - And then, we can use the request id to get the response body of a network call by using
devTools.send(Network.getResponseBody(requestId)).getBody().
devTools.addListener(Network.responseReceived(), responseReceived -> {
RequestId requestId = responseReceived.getRequestId();
System.out.println("Response body: "+ devTools.send(Network.getResponseBody(requestId)).getBody());
});
Whole Code
import java.util.Optional;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v108.network.Network;
import org.openqa.selenium.devtools.v108.network.model.RequestId;
public class CodekruTest {
public static void main(String[] args) {
// pass the path of the chromedriver location in the second argument
System.setProperty("webdriver.chrome.driver", "E:\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
DevTools devTools = ((ChromeDriver) driver).getDevTools();
devTools.createSession();
devTools.send(Network.enable(Optional.of(1000000), Optional.empty(), Optional.empty()));
devTools.addListener(Network.responseReceived(), responseReceived -> {
String responseUrl = responseReceived.getResponse().getUrl();
RequestId requestId = responseReceived.getRequestId();
if (responseUrl.contains("makemytrip")) {
System.out.println("Url: " + responseUrl);
System.out.println("Response body: " + devTools.send(Network.getResponseBody(requestId)).getBody());
}
});
driver.get("https://www.makemytrip.com");
}
}
We hope that you have liked the article. If you have any doubts or concerns, please write to us in the comments or mail us at [email protected]