Tag Archives: MGT

MGT Testlanguage

Part of the series about a project of mine: MGT

After I wrote about the metamodel for the tests in the last post, it’s easy to present the language now. The language is just an easy to write serialization of the model. The more interesting part is the tooling to derive the language from the model and an editor to use the language. So I start with this first and we’ll see the result at the end.

For deriving the language from the Eore model I used EMFText. For the basic installation and usage have a look at the projects homepage. To define the language one has to add a syntax description for any model element. The descriptions are like EBNF notations. The following snippet shows the ones of MGT:

    TestSuite ::= "TESTSUITE" #1 "{" !1
                        "suiteName"":" name[BIGNAME]"," !1
                        "systemUnderTest"":" systemUnderTest[BIGNAME]"," !1
                        "adapter"":" adapter[IDENTIFIER] !1
    TestCase ::= "TESTCASE" #1 "{" !2
                        "caseName" ":" name[IDENTIFIER] !2
                        (testSteps ";"!1)+ 
    Assertion ::= "ASSERT" assert[IDENTIFIER]"(" actual ("," expected)? ")" (":" 
                    verdict[fail : "FAIL", error : "ERROR", inconclusive : "INCONCL", pass : "PASS"])?;
    UIElementLocator ::= "id" id[IDENTIFIER];
    Keyword ::= name[IDENTIFIER]  (":" target)?  (":" testData ("," testData)*)?  ;
    Sequence ::= "SEQUENCE" #1 name[IDENTIFIER] #1"{" !1
                        (steps ";"!1)+ !0
    SequentialStep ::= "seq:" sequence[IDENTIFIER];
    Integer ::= dataValue[INTEGER];
    Boolean ::= dataValue["true" : "false"];
    String ::= dataValue['"','"'];
    Float ::= dataValue[FLOAT];

I don’t wanna explain the whole thing now so I refer again to the documentation of EMFText or you can compare it to the actual language below.
Besides the syntax you can also define the tokens that are allowed to fill the models properties (text in square brackets, e.g. IDENTIFIER), the token styles and some options for code generation. All that is done in one ‘concrete syntax’ file.

Having a valid cs-file you can start generating code. EMFText will generate three Eclipse Plugins. The first one contains Java-Classes that represent those of the model (EMF itself can also do this). Another Plugin is used to parse text files to models and vice versa. This plugin uses antlr to parse the text. The third plugin provides code to integrate the language into the Eclipse editor, like code completion, run configurations, code styles, validation and so on. To make it more usable one should adapt the third plugin, which I also did and will present in another post.

With those generated Plugins you can configure a new Eclipse environment that allows you to write instances of your metamodel. The following screenshot shows the MGT test (model) editor with validation (errors and warnings) and code completion.
As this is a complex topic don’t hestitate to ask and/or discuss.

Test Meta Model of MGT

Part of series about my thesis project MGT.

This post is about to explain the meta-model upon which the test language is based. I will explain the different parts and the reasons for this design. The model is based upon the concept of keyword-driven testing.
It should be said, that I know that nothing of this is perfect. At the end of the post I’ll mention the most critical problems.

Let’s start with a image of the meta-model. The meta-model can be split into 3 parts. Classes of the upper left part describe the structure of tests. In the upper right part classes for test actions are located. And the lower part has classes to describe test data.

The first part is pretty simple. I took a common approach to structure tests. Starting point of any model in MGT is a TestSuite. It has a name and two other properties, which are used to state which system should be tested and which adapter is used. TestSuites may contain TestCases and Sequences, which both have a name as property and can contain several (Abstract-) TestSteps. Of course TestCases are exectuable Tests. Sequences can be used to group common TestSteps that occur more than once in the suite (e.g a login procedure).

The next part describes the different TestSteps (actions) that can be used. There are three classes that extend the AbstractTestStep. The first one, SequentialStep is used to call a Sequence. This can be seen as an import of the TestSteps that are stated in the Sequence. To perform an action in the system under test Keywords must be used. To keep the metamodel light and flexible the abstract concept of keyword-driven testing is used. The degree of abstraction is left to the user. You can hide a simple click or a complex workflow behind a keyword. Keywords can reference different test data, which are described below.
The last possible TestStep is an Assertion. This one should be known too. It always references a keyword, which represents the actual part upon which the assertion is called and it may have a second argument that represents the expected state.

The last part left is about TestData. As the image above shows there are two kinds of test data. The UIElementLocator should be used to access GUIElements in an abstract way. You just state the ID of the element you want to work with. How exactly the ID looks like depends on the Adapters again. It can be used to access an ObjectMap or be the exact ID of an Android UI element.
Other TestData objects are encapsulated behind the abstract class TestData. Its subclasses are just simple data types as integer, boolean or string.

So this already was everything. The meta-model is pretty simple and therefore lightweight. The problem about hiding keywords, IDs, and others is that the user may not know them. But as the Adapters provide those data, they can be provided by the tooling. Another post about the frameworks architecture will describe this in more detail. Another problem I see, is the lacking support of more complex data types. As I said Keywords can be used to execute complex workflows. But those may also have more complex data types as the provided ones. One way to solve this problem would be to provide a concept of DataProviders. I also opened a task a Github for this, but it has no priority for now.

Finally it’s left to say that anyone reading this is invited to discuss about my work. Questions and criticism are welcome.

Introduction To MGT

Part of a series about the personal project MGT. See here for details.

The MGT project started as a thesis in 2012. MGT stands for Mobile GUI Testframework. The Framework contains a keyword based Test Language, an architecture, that uses Eclipse Plug-ins as Adapters for the different platforms and, well, the adapters. But let’s start at the beginning.

The thesis was motivated as follows. Compared to native desktop applications and also web applications mobile apps are distributed to a bigger count of platforms. The current hype of smartphones and tablets caused the creation of millions of apps. While there exist several tools for cross platform app creation (e.g. PhoneGap), testing is left out (as always). Looking at the app-stores shows that software quality is also an issue of mobile software development. Therefor the goal of my thesis was to develop a test language (incl. tooling) that enables cross platform testing.

In times of agile development processes it was also important that those tests could be executed automatically. Since any platform has its own test drivers the language is based on a meta-model so that adapters for the different platforms can be implemented. The meta-model was also a prerequisite for the inclusion in the ‘MATE’ project.

So the goal of the thesis finally was the creation of a test language based on a meta model and the automatic test execution of tests defined in that language.

At the end MGT was the outcome of the thesis. This post will give an overview about the result of the thesis. Details are provided in other posts.
The whole product is based on Eclipse. There is no real justification for that choice. I just knew it already and it’s widely used at my university. The following image shows the big picture of the tool.

Concrete Architecture

So I built the meta-model with EMF and generate the language through EMFText. Into the generated EMFText code I included an Adapter Interface as extension point, whose implementations provide platform-specific actions. Those actions are the provision of usable keywords and GUI elements and the test execution. The provided keywords and elements are used for code completion and validation of the test model. As an example and proof of concept I implemented that interface for Android. Usable keywords are those of the Robotium framework. GUI element ids are extracted from the styles folder of the app to test. For the test execution a test model is converted into a valid JUnit test file and executed through Androids instrumentation framework.

Posts about the meta-model, test-language and the Android Adapter will be written soon.