Built by Maven

Request headers

HtmlUnit mimics the browser as closely as possible, which includes sending request headers. You can customize headers at three different levels: Request level, WebClient level, and BrowserVersion level.

BrowserVersion level

To change request headers at the BrowserVersion level, create a customized browser version using BrowserVersionBuilder. 

final BrowserVersion browser =
    new BrowserVersion.BrowserVersionBuilder(BrowserVersion.FIREFOX)
          .setAcceptLanguageHeader("de-CH")
          .build();

final WebClient webClient = new WebClient(browser);

Methods available to customize basic browser behavior and request headers include:

  • setApplicationCodeName(String)
  • setApplicationMinorVersion(String)
  • setApplicationName(String)
  • setApplicationVersion(String)
  • setBuildId(String)
  • setPlatform(String)
  • setSystemLanguage(String)
  • setSystemTimezone(TimeZone)
  • setUserAgent(String)
  • setVendor(String)
  • setUserLanguage(String)
  • setBrowserLanguage(String)
  • setAcceptEncodingHeader(String)
  • setAcceptLanguageHeader(String)
  • setCssAcceptHeader(String)
  • setHtmlAcceptHeader(String)
  • setImgAcceptHeader(String)
  • setScriptAcceptHeader(String)
  • setXmlHttpRequestAcceptHeader(String)

WebClient level

To change request headers at the client level, use WebClient.addRequestHeader(). You can add additional headers to every request made by this client or overwrite default ones.

Example: Add an additional header to every client request: 

client.addRequestHeader("from-htmlunit", "yes");

Example: Replace the default accept-language header for all requests made by this client: 

client.addRequestHeader(HttpHeader.ACCEPT_LANGUAGE, "fr");

Request level

It is also possible to add or overwrite a request header for a specific request. 

WebRequest wr = new WebRequest(URL_FIRST);
wr.setAdditionalHeader("from-htmlunit", "yes");

client.getPage(wr);

Animations based on Window.requestAnimationFrame()

All browsers supported by HtmlUnit are able to run animations based on the Window.requestAnimationFrame() API. A typical example for this is Chart.js. Animation support is not triggered automatically because HtmlUnit runs headless. The JavaScript portion of the API is fully implemented, but users must explicitly trigger callback frames.

Example: 

try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) {
    HtmlPage page = webClient.getPage(uri);
    webClient.waitForBackgroundJavaScript(1_000);

    // page is loaded and async js done

    // process animation frames
    Window window = page.getEnclosingWindow().getScriptableObject();
    int i = 0; // limit animation cycles
    int pendingFrames = 0;
    do {
        i++;

        // force one animation cycle
        // invokes all animation callbacks registered for this window once
        pendingFrames = window.animateAnimationsFrames();
    } while (pendingFrames > 0 && i < 200);
}

This provides complete control over animation cycles—allowing you to skip frames or inspect page DOM state after each frame step.

File download and Attachments

Normally pages are loaded inline: clicking on a link loads the linked page in the current window. Attached pages are intended to be loaded outside of this flow: clicking on a link prompts the user to either save the file or open it externally without changing the active page.

HtmlUnit complies with attachment semantics when an AttachmentHandler has been registered via WebClient.setAttachmentHandler(AttachmentHandler). When no attachment handler is registered, attachments are loaded inline. By default, AttachmentHandlers are not registered on new WebClient instances.

Please find more details and samples in the File download section.

Clipboard

Clipboard interaction is disabled by default for WebClient. This avoids side effects during test execution and removes dependencies on a running graphical desktop environment (Windows/X11/xvfb).

To enable clipboard support, attach a clipboard handler to the WebClient. HtmlUnit provides AwtClipboardHandler for interacting with the system desktop clipboard (requires a graphical environment): 

final ClipboardHandler clipboardHandler = new AwtClipboardHandler();
webClient.setClipboardHandler(clipboardHandler);

You can then control clipboard content programmatically: 

clipboardHandler.setClipboardContent("HtmlUnit");

You can also implement a custom ClipboardHandler for full control or when running in headless environments.

Content blocking

Out of the box, HtmlUnit does not include default content blocking, but provides simple extension points to add custom blocking logic.

Blocking based on the request (URL)

This form of content blocking checks requested URLs against blocked lists or patterns. When blocked, the request is intercepted before hitting the network, returning a custom mock response.

Implement this with a WebConnectionWrapper: 

try (WebClient webClient = new WebClient()) {
    webClient.getOptions().setThrowExceptionOnScriptError(false);

    // create a WebConnectionWrapper with a custom getResponse() impl
    new WebConnectionWrapper(webClient) {

        @Override
        public WebResponse getResponse(final WebRequest request) throws IOException {
            final URL url = request.getUrl();

            if (!isBlocked(url)) {
                return super.getResponse(request);
            }

            // construct alternative response
            final String content = "<html></html>";
            final WebResponseData data = new WebResponseData(content.getBytes(StandardCharsets.UTF_8),
                    200, "blocked", Collections.emptyList());
            final WebResponse blocked = new WebResponse(data, request, 0L);
            blocked.markAsBlocked("Blocked URL: '" + url.toExternalForm() + "'");
            return blocked;
        }

        private boolean isBlocked(final URL url) {
            return url.toExternalForm().contains("ad-server");
        }
    };

    final HtmlPage page = webClient.getPage(url);
}

Blocking based on the response (headers)

Requires HtmlUnit 3.4.0 or later

Response-based blocking evaluates headers (such as Content-Length) before downloading full response payloads, allowing early cancellation to improve test performance.

Subclass HttpWebConnection to intercept headers prior to downloading the response body: 

try (WebClient webClient = new WebClient()) {
    webClient.getOptions().setThrowExceptionOnScriptError(false);

    webClient.setWebConnection(new HttpWebConnection(webClient) {
        @Override
        protected WebResponse downloadResponse(final HttpUriRequest httpMethod,
                final WebRequest webRequest, final HttpResponse httpResponse,
                final long startTime) throws IOException {

            final Header contentLengthHeader = httpResponse.getFirstHeader(HttpHeader.CONTENT_LENGTH);
            if (contentLengthHeader != null) {
                final int contentLength = Integer.parseInt(contentLengthHeader.getValue());

                // allow small downloads
                if (contentLength < 1_000) {
                    return super.downloadResponse(httpMethod, webRequest, httpResponse, startTime);
                }

                // abort body download for large content
                httpMethod.abort();

                // construct alternative response
                final String content = "<html></html>";
                final WebResponseData data = new WebResponseData(content.getBytes(StandardCharsets.UTF_8),
                        200, "blocked", Collections.emptyList());
                final WebResponse blocked = new WebResponse(data, webRequest, 0L);
                // if you like to check later on for blocked responses
                blocked.markAsBlocked("Blocked URL: '" + webRequest.getUrl().toExternalForm()
                            + "' content length: " + contentLength);
                return blocked;
            }

            return super.downloadResponse(httpMethod, webRequest, httpResponse, startTime);
        }
    });

    final HtmlPage page = webClient.getPage(url);
}

Blocking loading of frame content

By setting a custom FrameContentHandler on the WebClient, you can control whether specific frame documents should be loaded. 

try (WebClient webClient = new WebClient()) {
    webClient.setFrameContentHandler(new FrameContentHandler() {

        @Override
        public boolean loadFrameDocument(final BaseFrameElement baseFrameElement) {
            final String src = baseFrameElement.getSrcAttribute();
            // prevent loading frame content from external trackers
            return !src.contains("google");
        }

    });

    final HtmlPage page = webClient.getPage(url);
}

Multithreading / Thread Pooling

HtmlUnit uses an ExecutorService backed by a CachedThreadPool for background thread management (such as WebSocket and asynchronous processing).

You can provide a custom executor service using WebClient.setExecutor(ExecutorService) to manage thread pool sizing or apply custom thread naming conventions.

Local / Session Storage

HtmlUnit supports the Web Storage API. The storage structures are accessible directly from WebClient using StorageHolder to inspect or pre-populate data before test execution.

SessionStorage example

try (WebClient webClient = new WebClient()) {

    // get session storage for current window
    final Map<String, String> sessionStorage =
            webClient.getStorageHolder().getSessionStorage(webClient.getCurrentWindow());

    // pre-populate session storage
    sessionStorage.put("myKey", "myData");

    // load page that consumes session storage
    webClient.getPage(url);

    assertEquals("myNewData", sessionStorage.get("myNewKey"));
}

LocalStorage example

try (WebClient webClient = new WebClient()) {

    // get local storage for target URL
    final Map<String, String> localStorage = webClient.getStorageHolder().getLocalStorage(url);

    // pre-populate local storage
    localStorage.put("myKey", "myData");

    webClient.getPage(url);

    assertEquals("myNewData", localStorage.get("myNewKey"));
}

Client-side certificates

HtmlUnit optionally supports client-side SSL/TLS certificates and custom trust stores configured via WebClientOptions: