Automation with Selenium,Junit, Ant

What is Selenium
How does Selenium work
History and how it started
About Ant
And Junit

Much of the technologies above do not or will not need an introduction if you already know them or can read them from the links above.

More over today’s article is more about how we can use all the three selenium, ant and junit to come up with an automated solution for regressive testing.

Please refer to the documentation links above for the basic knowledge on any of the used tools.

I am assuming that you know how to record a test in selenium if you dont then go here

Now that you do know how to record and save the tests simply save them in any directory structure you like as long as they follow the right package convention. like com.foo.bar
this would mean the file should be in directory like com/foo/bar. Conventionally and logically today that is how the TestCase is compiled through the ant script. If the package is not the right path then compile time errors will occur.

My calling target for running the whole procedure should look like this

< target name="build_tests" depends="clean, compile, start-server,  tests, stop-server" / >

Details of the depending targets are as follows.
clean: would clean the entire build directories etc to make sure nothing from our past is carried forward into the future.

compile: should compile

start-server: should start the selenium server.

tests: should run the selenium tests.

stop-server: should stop the selenium server once all tests have been executed.

How to start a selenium server from Ant?

    < target name="start-server" >
        < java jar="lib/selenium-server.jar" fork="true" spawn="true" >
            < arg line="-timeout 30" / >
            < jvmarg value="-Dhttp.proxyHost=proxy.proxyhost.com" / >
            < jvmarg value="-Dhttp.proxyPort=44444" / >
        < / java >
    < / target >

 

The target above is simply taking the selenium-server.jar which is in the classpath and running the main-class from it. the proxy parameters are optional.

How to stop the server from Ant?

   
 < target name="stop-server" >
        < get taskname="selenium-shutdown" 
            src="http://localhost:4444/selenium-server/driver/?cmd=shutDown"	
            dest="result.txt" ignoreerrors="true" / >
        < echo taskname="selenium-shutdown" message="DGF Errors during shutdown are expected" / >
    < / target >
 

The selenium server starts on port 4444 by default and to tell it to shutdown is simple, the command in the src is passed to it. cmd=shutDown
That should just shutdown the server.

Executing the test cases?
Selenium Test cases are Junit tests cases and that’s how they are treated in this tutorial.
Thus some of you will be very much familiar with the ant targets junit and junitreport. However I will describe how the following is working.
the following target will run the selenium tests and print a summary report to the ${dir}
The includes and excludes, simply tell the target which files to include or exclude while running the test cases. This is typically done when you don’t want some tests cases to be included or your source for tests and the application are in the same directory and you only want to include something like Test*.class.

  < target name="tests" depends="compileonly" description="runs JUnit tests" >
    < echo message="running JUnit tests" / >
    < junit printsummary="on" dir=".." haltonfailure="off" haltonerror="off" timeout="${junit.timeout}" fork="on" maxmemory="512m" showoutput="true" >
      < formatter type="plain" usefile="false" / >
      < formatter type="xml" usefile="true" / >
      < batchtest todir="${testoutput}" filtertrace="on" >
        < fileset dir="${src}" >
          < includesfile name="${tests.include}" / >
          < excludesfile name="${tests.exclude}" / >
        < / fileset >
      < / batchtest >
      < classpath >
        < pathelement path="${classes}" / >
        < pathelement path="${build.classpath}" / >
      < / classpath >
    < / junit >

The following will take the formatted output from the lines above and generate a report out of it in xml and html and place the results in the ${reports}/index.html
A sample Junit test report might look like this.

   < echo message="running JUnit Reports" / >
   < junitreport todir="${reports}" >
      < fileset dir="${reportdir}" >
        < include name="Test*.xml" / >
      < / fileset >
      < report format="frames" todir="${reports}" / >
    < / junitreport >
    < echo message="To see your Junit results, please open ${reports}/index.html}" / >
  < / target >

In general you can add all of this to your nightly build through any of the CI servers like Cruise Control. Also as a general practice you will need to do a little more then just executing this target every night depending on your application. For instance cleaning up of resource centers like Databases etc.

Continous Integration – A blast from the past

Although this didn’t happen a decade ago but still has been a good case for me to learn and realize how Continuous Integration brings value addition to our work.

As I recall it was like this when they were teenagers 😀
Few teams working on different modules of same application, deployed together.
No build process formalized
No builds except for the ones that need major milestone deployments.
No feedback, reports, reviews etc.

So virtually there was no build eco system.

Which meant hideous amounts of communication, tons of trouble shooting, last minute show stoppers, And extermely unhappy end users/stake holders.

So what exactly was going wrong.

There was no Configuration Control i.e. No way to figure out the changes that will be part of the release and no controls to include or exclude them.
No Status accounting i.e. No feedback or status of anything as a whole product.
No management of Environment meant no one knew what hardware/software they were using and sometimes why?
No team work whatsoever meant no guided processes, reviews or tracebility of issues and pitfalls.

So virtually a release day was the doomsday!

It took some time for developers to realize that how important the following were.
* Configuration identification – What code are we working with?
* Configuration control – Controlling the release of a product and its changes.
* Status accounting – Recording and reporting the status of components.
* Review – Ensuring completeness and consistency among components.
* Build management – Managing the process and tools used for builds.
* Process management – Ensuring adherence to the organization’s development process.
* Environment management – Managing the software and hardware that host our system.
* Teamwork – Facilitate team interactions related to the process.
* Defect tracking – Making sure every defect has traceability back to the source

And after all it takes some time but the day to day flow looked like the figure below.

Build Management
Build Management

Importantly
Developers are distributed on different locations in different teams.
There is an SCM team that takes care of the SCM activities i.e. Configuration Identification, Control, Build Management etc.
There is one single repository for all developers.
A process is defined for Product integration and its checked whenever some one checks in.
And hourly build is dont to check the sanity of the system with the unit tests.
A Nightly build is done with all full blown unit tests, coverage reports, Automated tests and installers and upgrades.
More over all of the above is shown on an FTP/HTTP site. FTP for installer/binary/archive pickup and HTTP for detailed view of builds/feedbacks etc.

All of the above meant strength and health of the build eco system. And more trust in the system that it will report failures along the way.

That thing about CI – Continous Integration

Challenging Business requirements and the need for software development teams to remain agile and competitive while managing parallel development and releases requires a system which is adaptive to these demands.

Our approach to SCM enables unlimited and adaptable process models, which are ideally suited for parallel, distributed, and agile software development. Using state of the Art technologies from various vendors to automate processes such as branching, merging, build, and release keeps you a click away from software delivery.

The real matter then is the sanity of your application and that’s the challenge we all need to address. We can have cute products right of the shelf to get it up and going but the whole challenge lies in the statement “up and going”. It takes a ton of energy and effort to come up with the right process and the right tool. I have not seen alot of CI projects failing strictly. But I do have seen extra efforts into things that should not be really there. failure would then mean the amounts of efforts spent on doing not so trivial stuff and on the other end if you are spending alot of time doing the major flows then definately something is wrong.

Automation does solve most parts of the problems but things that really need to be taken care of to get it right in the sweet spot are the ones Martin Fowler described in one of his articles.

* Maintain a Single Source Repository
Keep everyone together, at the end of the day we all work together dont we?

* Automate the Build
Automation of the build helps you.

* Make Your Build Self-Testing
Self testing build tells you exactly what you did and what you broke. So I can come up with this statement to my Lead. “If I commit this, it might break alot of stuff”. Well Sir, try running the Unit Test, You will know for sure! But then again honesty on the test cases really doest matter!

* Everyone Commits Every Day
Typically in a Agile environment helps alot, and forces you not to break the build. Better organization I must say.

* Every Commit Should Build the Mainline on an Integration Machine
Sanity checking of your commits. So often we commit without realiazing that we break something else.

* Keep the Build Fast
I usually commit before leaving for home. It helps great deal when you get the feedback within 5 mins. kind of a benchmark but depends.

* Make it Easy for Anyone to Get the Latest Executable
Keep all the build results and one sharable place. We did that on one of the http servers some time back

* Everyone can see what’s happening
Importantly that all results are published for all of us to know.

* Automate Deployment
Deployment automation means that my buddy X on the other side doesnt screw up the server for even a 1 min. Only deploys when build successfull.

Hope this helps.

Resources:

http://martinfowler.com/articles/continuousIntegration.html

SVN – some quick commands during merge.

Following are some of the frequently used svn commands during merging and branching. I used to work with tortoise for this but as soon as I learned these, it feels like a more easier space to be in. Although no points taken away from tortoise, it still works pretty good for the gui part, This article is more targeted towards dark screen lovers.

Create a new branch from trunk:
If you want to create a branch from a specific revision of Trunk following command is handy. It does a remote copy. which means the machine you are on does not need a copy of the whole tree.

$ svn copy -r REVISION TRUNK_URL NEW_BRANCH_URL --username USERNAME --password PASSWORD -m MESSAGE

An example

$ svn copy -r 1234 http://shaafshah.com/trunk http://shaafshah.com/branches/MY_BRANCH --username foo --password bar -m "Remote copy"

When was this branch created?
If you want to know the day branch was created.

$ svn log -v --stop-on-copy BRANCH_URL

The last record will show you the day the branch as created.

List all the branches:
If you want to take a listing of branches or a tree

$ svn ls BRANCHES_URL

e.g. http://shaafshah.com/branches

Merge from Branch to Trunk:

Browse to where you have checkedout trunk in the local directory.

$ cd shaafshah.com/trunk

Update trunk to HEAD.

$ svn update
At revision 1234.

Following will merge from branch to trunk but will not commit.

$ svn merge -r BRANCH_REVISION:TRUNK_REVISION BRANCH_URL

Branch_REVISION will be the revision branch was created if this is the first time you are doing the merge.

$ svn merge -r 1233:1234 http://shaafshah.com/branches/my_new_branch

After this you should do an

svn status

to check the status of the files. the files will be marked with following Characters.

‘A’ Added
‘C’ Conflicted
‘D’ Deleted
‘I’ Ignored
‘M’ Modified

If there is any ‘C’ in the status the files will not be committed if you try an svn commit to save the merge to the trunk.

Merge from Trunk to Branch:

To merge from Trunk to branch you would need to browse to the branch checked out in the local direcotry.

$ cd shaafshah.com/branches/mybranch
$ svn update
At revision 1234.

The following command will try merging trunk from revision 1233 i.e. the day branch was created to branch head.

$ svn merge -r 1233:HEAD TrunkURL BRANCH_URL

Hopefully this should help. However you should definitely refer to SVN Book for more detail.

svnbook at http://svnbook.red-bean.com/

Creating an SVN wrap for your build using Ant

Today after along break I would jump right on to one of the interesting topics in my den these days.

One of my friends was lately troubled with doing some SVN stuff like merging etc. And a lot of people will agree with me on their first experiences. 🙂 I think.

While Automated builds take a lot of our time I thought I could plug in with some automated merging and a few other tasks. Its hard to go over all of that in one post but I will try putting in some basic stuff to get us started.

I call it the SVN Wrap.

Step 1.
Create a simple script file for wrapping svn and our environment.

A simple bat script could look like the following

@echo off
svn %*

However some people might want to add some environment variables to it. And that is where the strength of the this file comes in. You can tailor the environment dynamically!
e.g.

set LC_ALL=C
set SVN_HOME=svn-win32-1.4.6
set PATH=%SVN_HOME%\bin;%PATH%;

FYI: By setting LC_ALL I am telling the system I disregard the default locale. Its just used as an example here. for more information refer to the svnbook at http://svnbook.red-bean.com/

Step 2.
Create the build.xml
It doesnt get simpler then this.

I have created a project with the name CI-Test

<project name="CI-Test" default="status" basedir=".">
	<description>
		This is a POC for SVN Wrap.
	</description>
</project>

Importantly I am setting a property local.branch so that I can tell svn where my code has been checked out locally.

And finally the target that will take a status of the branch. for more details on the status command you could go here.

In general this target will give a general overview of the files and thier state at the moment.

	<target name="status">
		<echo message="Following is the status for this tree."/>
		<echo message="output is logged here: status.out" />
	          <exec dir="${local.branch}" executable="ci-svn.bat" output="status.out">
            		<arg line="status"/>
          	</exec>
	</target>

With in the target there are a few echo commands but the key construct is the exec.
The exec is going to do the following

dir=”${local.branch}” – would meant execute the command in this directory
executable=”ci-svn.bat” – Identifies the executable
and finally the output attribute to show where the output of this activity goes.
The following line will pass the parameters to the executable
< arg line=”status” />
And in this case its sending a status command to svn.

For more details on exec goto. http://ant.apache.org/manual/

Now I would presume both these files in saved in one direcotry as

1. ci-svn.bat
2. build.xml

To run this simply goto your command console and once in the same directory execute by running
ant

It is assumed that all paths to java,ant and svn are set on your console or system.

As an output you should be able to see a status.out in the same directory from where you executed ant.

Hopefully this should get you started with doing some bit of svn commands from ant. And that just opens a lot of more possibilities in your build environment.

The complete code listing of the build file is as follows.

<project name="CI-Test" default="status" basedir=".">
	<description>
		This is a POC for SVN Wrap.
	</description>
 
	<property name="local.branch" value="C:\branches\my-branch"/>
 
	<target name="status">
		<echo message="Following is the status for this tree."/>
		<echo message="output is logged here: status.out" />
	          <exec dir="${local.branch}" executable="ci-svn.bat" output="status.out">
            		<arg line="status"/>
          	</exec>
	</target>
 
</project>