Parallel Web Testing With A Queue Of Robot Users

I wanted to use Selenium to test some capabilities of this web application my team was building, so I wrote a lot of tests in Java and started running them. Okay, great, if any of these capabilities being tested break in the course of development, the robots will find out about it and report back to us. Nothing new here; that’s the nature of test automation. The same old story usually continues down a path like “These tests are great, but what can we do to optimize how they run?” For example if we want our full suite of tests to go through an end-to-end run in less time, we’ll consider running tests in parallel. And so that’s what we did.

There are a thousand and twelve benefits to running your UI tests in parallel, and it feels like there are just as many pitfalls. I’d like to address one of the pitfalls I encountered, and that was dealing with our application’s session management. This web application requires its users to log in in order to access any of the good test-worthy capabilities. When a single user logs in on Computer A, a session is created for that user — this is so all the parts of the application know that this user guy is authorized to see and use all these protected capabilities, at least until the session ends or expires. But then if the same user logs in on Computer B, a new session is created for that user and that first session is ended. So back on Computer A, the user suddenly loses access to all these protected capabilities. The end result is if we wanted to test two capabilities on different machines at the same time, we were going to need to do it with two different users. Substitute “x” for “two” where “x” is the number of tests we wanted to run in parallel, and you see the kind of problem we were trying to solve.

Here’s how we did it.

The plan was to set up a queue of valid test accounts. When a test script wanted to log into our application, rather than each using the same “userX” login, they’d grab the first test username in line, removing it from the queue. That way when the next test script wanted to grab a username of its own, it would get the new first-in-line username and everyone would have its own unique test user and it would be all smiles and magic and butterscotch valleys. When a test script was finished it would add its username to the back of the queue so some future script could reuse it.

First, we registered an army of test accounts in our application, with usernames in a series like this:

robot1, robot2, robot3 .. robot39, robot40

Since we have 40 robots, we can run up to 40 tests at a time without running out of valid usernames to use. For us, that’s plenty.

Now, before we set this up we were passing the username as a straight up string for Selenium to plop into the “login” field of our application:

String APP_USERNAME = “userX”;

and then…

driver.findElement(USERNAME_FIELD_ID).sendKeys(APP_USERNAME);

What we wanted to do was populate that value with a String popped off our queue:

APP_USERNAME = RoundRobin.getAppUsername(appUsernames);

and elsewhere:

public class RoundRobin {

private static Queue<String> appUsernames = new LinkedList<>();

public static Queue<String> setRoundRobin() {
 for(int i = 0; i < 40; i = i + 1){
   String thisSuffix = Integer.toString(i);
   String thisUsername = String.format(“robot%s”,thisSuffix);
   RoundRobin.appUsernames.add(thisUsername);
   }
 return appUsernames;
}

public static String getAppUsername(Queue<String> appUsernames) {
  return appUsernames.remove();
}

}

In RoundRobin, we’re setting up the Queue of appUsernames which will live in this class and will hold those 40 Strings, valid login values for our application. When we want to populate that Queue, before we start running all our tests, we have the setup class setRoundRobin fill it up with Strings and return the whole object so the setup class knows the deed is done. Here’s what the setup class looks like:

public class SuiteSetup {

 @BeforeSuite(alwaysRun = true)
 public void loadRoundRobin() throws InterruptedException {
   RoundRobin.setRoundRobin();
   }
 return;
 }
}

@BeforeSuite is a TestNG annotation that tells this class to run before any other tests are run. In order for the test classes to know that this is living out there, expceting to be run first, we have to make sure that all our test classes extend this setup class, like this:

public class VideoInProgressIndicatorTest extends SuiteSetup {

 private static String APP_USERNAME;
 private static WebDriver driver;
 
 @BeforeClass
 private static void startDriver() throws IOException {
   APP_USERNAME = RoundRobin.getAppUsername(appUsernames);
 }

 @Test
 private static void verifyInProgressIndicator() {
   driver.get(www.loginpage.foo);
   driver.findElement(USERNAME_FIELD_ID).sendKeys(APP_USERNAME); 
   test.allTheThings(driver);
 }
 
 @AfterClass
 private void teardown() {
   RoundRobin.returnAppUsername(APP_USERNAME);
   driver.quit();
 }
}

You’ll notice the ‘returnAppUsername’ method used in teardown — that’s what adds the username String to the back of the line for re-use. In RoundRobin it looks like this:

public static boolean returnAppUsername(String returningUsername) {
 return appUsernames.add(returningUsername);
 }

And there you have it. With this little trick we’ve been able to winnow down the number of pitfalls related to parallel testing to a lean one thousand and eleven.

pitfalls--;