Introduction
The WebClient represent the browser if you work with HtmlUnit. To start using HtmlUnit you have to in instantiate a new WebClient - like starting the browser in the real world.
try (final WebClient webClient = new WebClient()) { // no you have a running browser and you can start doing real things // like going to a web page final HtmlPage page = webClient.getPage("https://htmlunit.sourceforge.io/"); } }
Imitating a specific browser
Often you will want to simulate a specific browser. This is done by passing a com.gargoylesoftware.htmlunit.BrowserVersion into the WebClient constructor. Constants have been provided for some common browsers but you can create your own specific version by instantiating a BrowserVersion.
@Test public void homePage_Firefox() throws Exception { try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) { final HtmlPage page = webClient.getPage("https://htmlunit.sourceforge.io/"); Assert.assertEquals("HtmlUnit – Welcome to HtmlUnit", page.getTitleText()); } }
Specifying this BrowserVersion will change the user agent header that is sent up to the server and will change the behavior of some of the JavaScript.
Using the options to adjust the browser
There are various options available to make fine grained adjustments to the browser.
@Test public void homePage_Firefox() throws Exception { try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) { // disable javascript webClient.getOptions().setJavaScriptEnabled(false); // disable css support webClient.getOptions().setCssEnabled(false); final HtmlPage page = webClient.getPage("https://htmlunit.sourceforge.io/"); Assert.assertEquals("HtmlUnit – Welcome to HtmlUnit", page.getTitleText()); } }
The default values for most options are similar to the default values
of real browsers - but (as always) there is one important exception:
HtmlUnit stops the Javascript execution at the first unhandled exception -
Browsers do not stop. You can change this by changing the throwExceptionOnScriptError
option to false.
@Test public void homePage_Firefox() throws Exception { try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) { // procced with the js execution on unhandled js errors webClient.getOptions().setThrowExceptionOnScriptError(false); final HtmlPage page = webClient.getPage("https://htmlunit.sourceforge.io/"); Assert.assertEquals("HtmlUnit – Welcome to HtmlUnit", page.getTitleText()); } }
For a complete list and more details please have a look at the WebClientOptions API.
Using handlers
There are many handlers used by the WebClient for special purposes. These Handlers are implementing specific interfaces and you are able to replace them with your own implementations. Default implementations are also available.
AlertHandler
The handler to be used to process JavaScript alerts triggered when the JavaScript method Window.alert() is called.
ConfirmHandler
The handler for the JavaScript function window.confirm()
.
PromptHandler
The handler for the JavaScript function window.prompt()
.
StatusHandler
A handler for changes to window.status
.
AttachmentHandler
A handler for attachments, which represent pages received from the server which contain Content-Disposition=attachment headers.
ClipboardHandler
A handler for clipboard access.
PrintHandler
A handler for providing Window.print() implementations.
WebStartHandler
A handler for webstart support.
FrameContentHandler
A handler to make a decision to load the frame content or not.
AppletConfirmHandler
Like in real browsers, you have to confirm before the browser starts an applet.
CSSErrorHandler
For CSS parser error processing.
OnbeforeunloadHandler
RefreshHandler
A handler for page refreshes.
Polyfills
The number of javascript API's supported by the browsers seems to increase every day. Because of the limited development resources
of the HtmlUnit projcet, being on track with this is really hard.
But there are alrady many polyfills available (to add API support for older borwsers). The idea is to use some of these polyfills
to add the missing API's.
Starting with version 2.59.0 HtmlUnit supports the integration of polyfills; there is a dedicated option for every supported
polyfill (disabled per default) and if enabled, the polifyll is automatically loaded.
@Test public void fetchSupport() throws Exception { try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) { // enable fetch api polyfill webClient.getOptions().setFetchPolyfillEnabled(true); final HtmlPage page = webClient.getPage(....); } }
Fetch API Polyfill
webClient.getOptions().setFetchPolyfillEnabled(true);
Proxy Polyfill
webClient.getOptions().setProxyPolyfillEnabled(true);