As we start to assemble the tools we need to implement Continuous Delivery, one area we need to look at is selecting a testing framework. Now, we are going to want to use a range of different testing tools, depending on what we want to test and how, but we ideally would have some kind of framework that helps to abstract those tools and gives us a single place to go to execute tests.
In looking for a test framework, I have been considering the following:
- Preferably it should have some kind of backing to ensure its longevity and continued development,
- Ideally I should not have to learn a new language to use the tool – since most Fusion Middleware users are already using Java and Python (for WLST scripts) – those two seem like the best choices,
- It should be extensible, so I can add new types of test tools if I want to,
- It should integrate well with Maven and Hudson, since those are already part of our ecosystem,
- It should allow me to use a wide range of testing tools to execute the actual tests.
So after a bit of research, what I have come up with is Robot Framework. It is backed by Nokia Siemens Networks, it uses a simple text/table based test specification approach, Python and Java are supported for extension, there are Maven and Hudson plugins, and there are a wide range of plugins for many common testing tools like Selenium, Swing, iOS, Android, and support for OS integration (processes), XML, HTTP, SOAP, etc.
Seems like a good choice. So let’s take a look at it!
In this earlier post, we created a basic web application from the WebLogic Server ‘Basic WebApp’ Maven archetype. Let’s take that application and add some tests to it. We will do this using the Robot Framework Maven Plugin and the Selenium Library.
So the first thing we want to do, is add the Robot Framework Maven Plugin to our pom.xml by adding a new plugin element to the build section. Here is what we need to add:
<plugin> <groupId>org.robotframework</groupId> <artifactId>robotframework-maven-plugin</artifactId> <version>1.2</version> <executions> <execution> <id>robot</id> <phase>integration-test</phase> <goals> <goal>run</goal> </goals> </execution> </executions> </plugin>
This tells Maven that we want to run the Robot Framework during the integration-test phase of the build. That is all the configuration we need. As with most things in Maven, there are a set of sensible defaults that we can follow to avoid any additional configuration burden. In this case, we will take advantage of two – the Robot Framework plugin looks for libraries in src/test/resources/robotframework/libraries and tests in src/test/robotframework/acceptance.
We are going to use the SeleniumLibrary. You can download the library from here. Inside the download you will find a directory called robotframework-seleniumlibrary-2.9.1/src/SeleniumLibrary. You need to copy that directory (i.e. SeleniumLibrary) into your project under src/test/resources/robotframework/libraries.
You can find documentation for this library here.
Now we can write a test. To do this, we just need to create a file in src/test/robotframework/acceptance. Let’s call it Basic_Test.txt. Here is how we specify the test using Robot Framework:
*** Settings *** Library SeleniumLibrary Test Set Up Start Selenium Server Test Tear Down Stop Selenium Server *** Test Cases *** Basic Test Open Browser http://localhost:7001/basicWebapp/ Page Should Contain Basic Webapp Page Should Contain Textfield j_idt10:name Page Should Contain Textfield j_idt10:amount Page Should Contain Button j_idt10:j_idt18 Input Text j_idt10:name Fred Input Text j_idt10:amount 39.99 Click Button j_idt10:j_idt18 Wait Until Page Loaded Page Should Contain The money have been deposited to Fred, the balance of the account is 39.99 Close Browser
Let’s take a look at how this test is specified. Firstly, you will see that the format is reasonably simple. Sections are marked with headings which start and end with three asterisks. This test has two sections:
- Settings – tells Robot Framework about any libraries we want to use and any global actions to take. In this case, we want to use the SeleniumLibrary and we want to start the Selenium server before running the tests, and then stop it after all tests.
- Test Cases – this section lists all of the test cases. In this example we only have one, called Basic Test, but we could have as many as we like listed here.
Test Cases are written using keywords. The keywords are the first thing on each line, for example Open Browser, Page Should Contain, and so on. After the keywords, there are (sometimes) parameters.
Robot tests are written in files like this. You do not actually have to line everything up in a table as shown here, the minimum is three spaces before the keyword and two spaces between the keyword and each parameter.
You can look up details of what each keyword does, and what others are available in the library documentation.
Keywords like Click Button take as a parameter something called a ‘locator’ – simply put these can be the id or name attributes of the HTML tag.
So this test will do the following:
- Open a browser and go to the application
- Check that the page shows up correctly
- Fill in some data and take an action
- Check that the expected outcome occured
To run the tests, all we need to do is execute Maven:
In the output you will see the tests running and output like this:
[INFO] --- robotframework-maven-plugin:1.2:run (robot) @ my-basic-webapp --- ============================================================================== Acceptance ============================================================================== Acceptance.Basic Test ============================================================================== Basic Test | PASS | ------------------------------------------------------------------------------ Acceptance.Basic Test | PASS | 1 critical test, 1 passed, 0 failed 1 test total, 1 passed, 0 failed ============================================================================== Acceptance | PASS | 1 critical test, 1 passed, 0 failed 1 test total, 1 passed, 0 failed ==============================================================================
Robot also produces a report which you can view in your browser:
So that is a gentle introduction to Robot Framework. In future posts we will see how we can expand its usage to orchestrate complex acceptance tests, and how we can write our own keywords and extensions.