Real Life Web App Integration Testing (IT) with Spring
I know what you’re thinking: “what constitutes ‘real life’ integration testing (IT)?” In my opinion, “real life” means “complicated web application” or maybe “web app that is stuck to creaky database not entirely under my control” or “web app that uses auditors to enforce security” or, probably most inclusively “web app that leverages a lot of Spring”.
I'm sure you know what I mean: the kind of application they pay you to work on and, clearly, not something you designed from scratch to be relatively clean and modular. Not your own pet project.
Table of Contents
Doesn’t Spring Have Awesome IT Support Already?
Spring does have some good IT tools in place already, things like the
TestContext framework and some special transaction management that
will roll back any database changes that your test may create. It
works pretty well but not for all cases; it really works best when
you’ve planned for it ahead of time and often not so well if you’re
only thinking about it after a good size of the code base is in place.
I think there are some positive points to the Spring approach but I’d argue that when your doing your integration testing, it’s kind of an oxymoron to test your components in isolation.
These are the issues I’m going to attempt to address. I think there are some positive points to the Spring approach but I’d argue that when your doing your integration testing, it’s kind of an oxymoron to test your components in isolation.
Our Test Project
Our test project will be simple but we’ll pull in a good portion of Spring’s features. We’ll use Spring Data with JPA for persistence and Apache Wicket to provide our web application. The application itself is pretty simple, it provides a simple to-do list and it lets you add "contexts" (the GTD-ism) as well as add and remove to-do items. You can take a look at the test project over on GitHub.
We’ll use Selenium with Ghostdriver and PhantomJS for our integration testing. Selenium will provide a nice Java API for managing the web browser and PhantomJS gives us a fast, headless browser. In my opinion, if you would also like to test real browsers you’re better off making your continuous integration environment perform those tests. You don’t want your developers bogged down by a lengthy and time consuming integration test phase; PhantomJS lets them run the IT tests as fast as possible.
The POM.XML File
As you might have guessed there’s a lot of good stuff setup in this project’s POM file. We’ll go over the parts relevant to IT testing to make sure you don’t miss anything.
Dependencies
We need to add JUnit, Selenium and Ghostdriver dependencies to our project. This is pretty straightforward.
<dependencies>
...
<!-- Testing -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.31.0</version>
</dependency>
<dependency>
<groupId>com.github.detro.ghostdriver</groupId>
<artifactId>phantomjsdriver</artifactId>
<version>1.0.3-dev</version>
</dependencies>
We pull in JUnit, Selenium and Ghostdriver (the PhantomJS driver for Selenium). You’ll need to have PhantomJS installed already, as long as it’s in your current path you won’t need to do any configuration at all.
A Note About PhantomJS
As of this writing, there is a known bug in the version of Ghostdriver bundled with PhantomJS 1.9.0; it has some issues switching over to an Iframe of content. Our test project uses Iframes for modal pop-up windows so you’ll want to build your own PhantomJS with the newest version of Ghostdriver, it’s all clearly documented on the Ghostdriver project page. If you’re on 64-bit Windows you can also use the one that I compiled.
The Jetty Plugin
When we run our integration tests we need Jetty to startup and load in our application. It needs to stay running while our tests run and it needs to cleanly shutdown when our tests are over. The Jetty Maven Plugin makes it easy to do just this. First, we set some properties up to let Jetty know what ports to use.
<properties> <jetty.stopKey>stopJetty</jetty.stopKey> <jetty.stopPort>8081</jetty.stopPort> <jetty.port>8080</jetty.port> </properties>
Here we tell Jetty what port to listen to for stop requests and which port it should use to provide our application.
Next we setup the plugin itself.
<build>
...
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>8.1.8.v20121106</version>
<configuration>
<webApp>
<contextPath>/</contextPath>
<descriptor>src/main/webapp/web.xml</descriptor>
<resourceBases>
<dir>src/main/webapp</dir>
<dir>src/test/webapp</dir>
</resourceBases>
</webApp>
</configuration>
In the stanza above, we add the plugin to our project and tell it where to find our web application…
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<daemon>true</daemon>
<stopKey>${jetty.stopKey}</stopKey>
<stopPort>${jetty.stopPort}</stopPort>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
<configuration>
<stopKey>${jetty.stopKey}</stopKey>
<stopPort>${jetty.stopPort}</stopPort>
</configuration>
</execution>
</executions>
</plugin>
</build>
Then we set Jetty to run right before our integration tests and to stop when the tests are complete. In this setup, Jetty will run in “daemon” mode so that Maven can move on and run our tests.
Tell Surefire to Skip Our IT Tests
We don’t want the regular Surefire plugin to run our tests during the regular “test” phase, we need to explicitly tell Surefire to skip them. This also goes in the “build” section of our POM.xml file.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/integration/**/*.*</exclude>
</excludes>
</configuration>
</plugin>
Tell Failsafe to Run Our IT Tests
The Failsafe plugin will actually run our tests. This plugin is very similar to Surefire, it justs runs the test at “integration test” time (the “verify” phase”) instead of during the “test” phase.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<includes>
<include>**/integration/**/*.*</include>
</includes>
</configuration>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
There’s not too much happening here, we setup a new “goal” called “integration-test” and we set this goal to target the “verify” phase. If any of our IT tests fail, Failsafe will ensure that the entire build fails as well.
Phew! That’s a lot of stuff.
The Test
With all of the infrastructure in place, we’re ready to write up our first IT test. I’ll go through the important bits next, you can look at the whole test over here.
Instantiate a New PhantomJS Before Each Test
Creating a new PhantomJS instance is pretty cheap so we’ll create a new one before each test is run.
private WebDriver driver = null; @Before public void setUp() { DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); driver = new PhantomJSDriver(desiredCapabilities); driver.manage().window().setSize(new Dimension(800, 600)); driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); }
Note that we set the size of the browser window, if you don’t explicitly set this it will be far too small.
Also interesting is how Selenium finds elements on the page. Whenever you tell it to find something on the page, it will scan through the DOM of the page until it finds the element you want and it will then return the found element or “null”. If you tell Selenium to “implicitly wait”, it will scan the page continuously for the element until it either finds it or the wait time is exceeded. This is a great fit for pages that have dynamic content, Selenium will scan the page until the element appears.
The Simple Case: Ensure the Page Renders
Here’s a very simple test that loads in a page and checks for a couple items of content.
@Test public void testHomePageRenders() { driver.get("http://localhost:8080/"); Assert.assertNotSame(driver.findElements(By.id("title")).size(), 0); Assert.assertNotSame(driver.findElements(By.id("addTodoLink")).size(), 0); Assert.assertNotSame(driver.findElements(By.id("addContextLink")) .size(), 0); }
Pretty simple! Selenium loads the page and then searches for some elements. We then check to see that the number of returned elements is not zero.
The Tricky Case: Fill Out a Modal With a Form
Now the tricky one, we want Selenium to load the home page and click on the button that creates a new “context”. When the modal pop-up window appears we’d like the form filled out with some test data and then have Selenium click on the form’s “save” button. Lastly we’d like Selenium to make sure that the new context appears on the home page. These are all AJAX requests, our to-do application is a single page application.
@Test public void textAddContext() { driver.get("http://localhost:8080/"); // get all of the current context identifiers final List<String> contextIdList = new ArrayList<String>(); for(WebElement context : driver.findElements( By.cssSelector(".context"))) { contextIdList.add(context.getAttribute("id")); } // open the "add context" modal driver.findElement(By.id("addContextLink")).click(); Assert.assertNotSame(driver.findElements( By.className("wicket_modal")).size(), 0); // switch focus to the modal WebElement webElementModal = driver.findElement( By.className("wicket_modal")); logger.info("Modal: " + webElementModal + ", " + webElementModal.getAttribute("id")); driver.switchTo().frame(webElementModal); logger.info("Focused on modal window"); // verify form elements rendered, fill out the form String contextName = "Test Context (" + (new Date().getTime()) + ")"; Assert.assertNotSame(driver.findElements(By.id("contextAddNameField")) .size(), 0); driver.findElement(By.id("contextAddNameField")).sendKeys("Test Context"); Assert.assertNotSame(driver.findElements( By.id("contextAddDescriptionField")).size(), 0); driver.findElement(By.id("contextAddDescriptionField")) .sendKeys("For all the testing"); Assert.assertNotSame(driver.findElements( By.id("formCancelLink")).size(), 0); Assert.assertNotSame(driver.findElements( By.id("formSubmitLink")).size(), 0); // submit the form driver.findElement(By.id("formSubmitLink")).click(); // switch focus back to the main window driver.switchTo().defaultContent(); // wait for the new context to appear WebElement webElementContextNew = (new WebDriverWait(driver, 30)).until( new ExpectedCondition<WebElement>() { @Override public WebElement apply(WebDriver webDriver) { // holder for the new context element WebElement webElementNew = null; // loop through all of our context identifiers for(WebElement context : webDriver.findElements( By.cssSelector(".context"))) { if(!contextIdList.contains(context.getAttribute("id"))) { logger.info("New context found with ID " + context.getAttribute("id")); webElementNew = context; } } return webElementNew; } }); Assert.assertNotNull(webElementContextNew); }
You can see that there’s a lot more going on here. In order to make this test more robust, we’re making a list of the current contexts before we add our new one. After we submit our form, we know our new context has appeared on the page when the list of contexts grows by one element.
The nice thing about Selenium is that the code is pretty easy to follow, it’s just more Java code. In my opinion the trickiest part is figuring out which parts of the page you need to wait for and then writing the right code to test the change.
The Idealized Case: You Just Saw It and Your App Isn’t It
It’s important to note that the example above covers a pretty idealized case, the same sort of application that most of the Spring with IT documentation spends its time on. It’s all pretty straightforward and it all works really great. Still, before we move on lets recap the assumptions we’ve made so far:
- We don’t need any fixture data and our tests will interact only with the web application. There’s no access to our Spring service or data layers at all.
- We’re using an ephemeral data store and we’re not worried about cluttering it up with stuff. In the above example we didn’t talk about it but it’d be reasonable to use an in-memory database.
Idealism, like it or not, is a exclusionary. By assuming a brand new application and presuming minimum complexity we’ve unwittingly excluded most real-life applications; adapting the code above to your own existing project may not work out so well. Such is life.
Access to the Spring Container
It doesn’t need to be this way, let’s address the first item: we need access to the Spring container in order to create and delete fixture data for our tests.
The first thing we need to do is to start using the
SpringJunit4ClassRunner, this will instantiate a Spring container
and then run our IT suite in the context of this container. Simply
adding an annotation to the top of the test class and then providing a
ContextConfiguration that points to our Spring configuration will be
enough to get this working.
@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = {"classpath:/applicationContext.xml"}) public class ToDoHomePageTest { // test code goes here... }
If you try to do this with our test application, you’ll see that this is a little more complicated than it looks. In our case we’re using an embedded Derby instance in our web application and when we try to spin up a second Spring configuration it fails because the files that Derby is using are locked. This isn’t a failure on the part of Spring, it’s just the way it is: you probably don't want to use an embedded database when your doing this kind of testing. You could fudge it by having the test Spring instance connect to the running Derby instance but we’re not going to go over that here, this article is already way too long as it is.
Transactions That Persist
Let’s assume that we’ve moved to a standalone database that already contains our test data, the next problem we’ll encounter is that no matter how we annotate our tests, the data never makes it to the database. This is a feature of the Spring JUnit runner: the transaction manager provided rolls back the transaction once the test completes. This is a nice idea but in the context of IT tests, not super useful. After all, we need our fixture data to make it into the database so that our web application can see this data and our tests can interact with it.
The fix here is easy, just ask Spring for a real transaction manager
and use the TransactionTemplate to get a real-life transaction for
your operation. Also, use the TransactionConfiguration annotation to
set the “defaultRollback” property to false.
@TransactionConfiguration(transactionManager = "txMgr", defaultRollback = false) public class ToDoHomePageTest { @Autowired private PlatformTransactionManager platformTransactionManager; @Autowired private ContextService contextService; private TransactionTemplate transactionTemplate; private Context context = null; @Before public void setUp() { transactionTemplate = new TransactionTemplate( platformTransactionManager); context = transactionTemplate.execute( new TransactionCallback<Context>() { @Override public Context doInTransaction( TransactionStatus transactionStatus) { Context context = new Context(); context.setName("Test Context"); context.setDescription("For all the testing."); return contextService.save(context); } } } @Test public void testAddToDo() { // test code here } @After public void tearDown() { transactionTemplate.execute(new TransactionCallback() { return contextService.delete(context); }); } }
We’ve added an AutoWired field for the transaction manager and the
context service. With these in hand we can go ahead and create our
fixture data, run our test and then delete the data when the test are
complete. As an added bonus, this will also let you test with
Hibernate's Auditors.
Where Are My Properties?
So far so good but if you’re loading data from *.properties files
you’ll notice that those files aren’t picked up by the test
runner. Lucky for you there’s another annotation for that, it may be
placed above your class declaration.
@PropertySource({"classpath:spring.properties", "classpath:other.properties"})
This will read in those two files of properties and make those properties available to your test Spring configuration.
Two Incompatible Beans?
The last gotcha' I've seen appears when your test Spring configuration is starting up, it complains that there is more than one bean available but that they are incompatible and thus the field can't be autowired. This one took me a while to track down, in my case there were mock service beans being instantiated to support regular unit tests and because Failsafe pulls all of the test classes into the classpath, these were clashing when the IT tests ran. The solution is simple: add an exclusion for these test classes in your Spring configuration file.
<context:component-scan base-package="com.windsorsolutions.todos">
<context:exclude-filter type="regex"
expression=".*ContextUnitTestConfiguration"/>
</context:component-scan>
The regular expression above excluded these mock beans and everything started working.
Your Mileage May Vary
My hope is that this article covers most of the issues you may run into when running IT tests against your Spring project. Still, Spring provides a lot of stuff and there may be issues lurking out there. If you run into anything really interesting, let me know and I'll add it to this article!
Comments powered by Disqus.