Tuesday, September 23, 2008

Eclipse GUI Testing Is Viable With SWTBot

For several years I've struggled with the viability of automated Eclipse GUI test frameworks. Now for the first time I've found an approach that works reliably. This article discusses the approach and details some specific techniques that greatly increase productivity in creating useful tests.

In the past automated GUI test frameworks have failed to meet my needs because they have focused too much on recording and playback. The complexity of the Eclipse environment makes it near impossible to get this approach to work reliably. Simple things such as background jobs and processing become major issues. Testing of GEF-based editors is not possible since all record/playback frameworks rely on identifying controls and widgets -- which GEF does not use.

Recently I've come across SWTBot, which uses a novel approach. SWTBot tests are written in Java, and run inside the Eclipse process as an Eclipse test application. This gives SWTBot full access to the SWT and Eclipse APIs. SWTBot tests are written as JUnits, which makes integration with common technologies such as Ant, Continuous Integration and code coverage tools easy.

But all is not rosy when writing tests with SWTBot. There are some Eclipse-specific idiosyncrasies that come back to bite you such as context menus that are reconstructed when shown and the GEF framework that does not use controls or widgets. That said, SWTBot provides and excellent starting point. Here's how I built on SWTBot to create a powerful test environment that is easy to use.

Initially my tests contained code that looks like this:


public void resetPerspective() {
bot.menu("Window").menu("Reset Perspective...").click();
bot.shell("Reset Perspective").activate();
bot.button("OK").click();
}

Not bad! It's easy to see what's going on. In other cases the code looked more like this:


SWTBotTree tree = bot.tree();
String[] path = name.split("/");
SWTBotTreeItem[] items = tree.getAllItems();
SWTBotTreeItem selectedItem = null;
for (SWTBotTreeItem item: items) {
if (path[0].equals(item.getText())) {
item.expand();
sleep();
selectedItem = item;
break;
}
}
for (int x = 1;selectedItem != null && x

As you can see, it's hard to see the forest for the trees. Way too much code is required to do simple things.

Here's what I did to make things easier:

1. Create classes that directly model the UI parts that you're working with in your tests. For example, instead of directly using SWTBotView to manipulate the Package Explorer view, create a class called PackageExplorer that delegates to SWTBotView and provides richer functionality. For example, in my PackageExplorer class I have the following method:


/**
* select the first element that adapts to the given resource
*/
public void select(final IResource resource)

2. Create a method to find and click context menu items in one go. In SWTBot the finding and clicking occur in two seperate UI runnables. In Eclipse this can cause problems for some context menus as the menu item gets disposed before it is clicked due to a loss of focus.

3. Instead of using sleep() to wait for something to be done, wait for the real thing to be done. A heavily loaded machine can cause processing times to vary. Instead of having a fragile sleep(500L), use a reliable technique to determine when the job is really done.
For example, if you know that your processing is holding a resource lock, post a no-op empty workspace job and wait on it inside your test. It will only be invoked once all other resource locks are released, so when it's complete you're guaranteed that your other job is done:


// ensure that all queued workspace operations and locks are released
ResourcesPlugin.getWorkspace().run(new IWorkspaceRunnable() {
public void run(IProgressMonitor monitor) throws CoreException {
// nothing to do!
}
}, new NullProgressMonitor());

4. Leverage and extend the SWTBot framework with Eclipse-specific behaviour. For example, make use of SWTBot's conditional waiting APIs by creating Eclipse-specific conditions like this one that is used to wait until an editor is opened on a resource:


/**
* a condition that is used to wait for an editor to open on a specific file.
*
* @author dgreen
*/
public class EditorOpenCondition extends DefaultCondition {
private final IFile file;

public EditorOpenCondition(IFile file) {
this.file = file;
}

public String getFailureMessage() {
return String.format("Timed out waiting for editor on %s to open",file.getFullPath());
}

public boolean test() throws Exception {
if (!file.exists()) {
return false;
}
return UIThreadRunnable.syncExec(new UIThreadRunnable.BoolResult() {
public boolean run() {
IEditorReference[] editorReferences = PlatformUI.getWorkbench().getActiveWorkbenchWindow().getActivePage().getEditorReferences();
for (IEditorReference reference: editorReferences) {
try {
IEditorInput input = reference.getEditorInput();
if (input instanceof IFileEditorInput) {
IFileEditorInput editorInput = (IFileEditorInput) input;
if (editorInput.getFile().equals(file)) {
return true;
}
}
if (input instanceof IStorageEditorInput) {
IStorageEditorInput editorInput = (IStorageEditorInput) input;
IPath fullPath = editorInput.getStorage().getFullPath();
if (fullPath.equals(file.getFullPath())) {
return true;
}
}
} catch (PartInitException e) {
e.printStackTrace();
} catch (CoreException e) {
e.printStackTrace();
}
}
return false;
}
});
}
}

5. Create a Java-based DSL for often-repeated use of editors or views. For example, for a static class diagram editor you might end up with a DSL that could be used as follows:


DomainModelDsl domainModelDsl = new DomainModelDsl();
domainModelDsl.create(project, "domain.dm");

domainModelDsl.
createEntity("A").
createEntity("B");

domainModelDsl.entity("A").extension("B");

Inside the DSL implementation the dirty work of manipulating the editor occurs. This makes it really fast to create complex tests that cover lots of ground.

6. Create GEF EditPart wappers. For GEF-based editors SWTBot doesn't give a lot of lift. For these you'll need to create classes similar to those provided by SWTBot, but instead of being widget-focused, they'll need to be EditPart-focused.

All in all I'm very impressed with SWTBot, which has finally delivered a viable automated GUI test framework for Eclipse and Eclipse RCP applications. I'm pleased to see that SWTBot has made an Eclipse project proposal, which if approved will hopefully lead to continued improvements and community adoption.

7 comments:

KetanPadegaonkar said...

Reg the expand-a-node snippet, you'll be happy to know that the tree api is chained, so you can do:
tree.expandNode(nodeText).select()

For the editor stuff:
you can instead just do:
bot.editor(fileName) which does an implicit wait, or a wait on: bot.activeEditor() or iterate on:
bot.editors()

Also any reason you're not on planet eclipse ? You've got some nice blogs on embedding images in the editor and stuff. I'm sure a lot of folks would love to read those.

David Green said...

Ketan, thanks for the tips and the great framework!

The bot.editors() call fails for me because the SWTBotEclipseEditor assumes that every editor has a styled text. A GEF editor has only a Canvas.

As for planet eclipse I've just requested that they put me on.

Regards, David

Andrew said...

Hi David

Great post and suggestions. I was wondering if you'd consider making more of the code you wrote available, or contributing it back to SWTBot? I am interested in using SWTBot for testing a GEF-based application.

Thanks,
Andrew

David Green said...

Andrew, thanks for the comment. I have indeed contributed back to the project. See bug 113 for details.

I'm sure with the move to Eclipse, Ketan is very busy. I'm not sure if he's had time to include my contribution or not.

My contribution did not adhere to the coding standards of SWTBot. It will likely need some massaging to fit in. Perhaps you could take it and work with Ketan to ensure that it meets the needs of the SWTBot project. You can get the relevant code here

Andrew said...

Sure David, I'm happy to help out where I can to get your contribution into SWTBot. Will get in touch with Ketan about this.

Anonymous said...

Hi Ketan,
Is the bug with the context menu already resolved?I mean the case when some sub-commands need to be find and clicked.

David Green said...

@Anonymous your best bet for having SWTBot questions answered is via the SWTBot Community Forum