Tag Archives: software testing

Standard

Posted by

Posted on

April 3, 2012

Posted under

Software testing

Comments

6 Comments

One year as a test consultant – a retrospective

Throughout my career I have mostly worked in house for a “software house”, with which I mean an organisation that builds software as a core of their business model. Even my times at Finalist IT Group and Quantiq X-Media, when working for external customers, and in the case of Finalist on site at the customer, I have always worked for organisations that create software to sell or sell the services of developers.

When I left Spil Games I decided I wanted a change in my career? or better said, i wanted to try different side of the business. I had spent the better part of the last 4 years managing people and processes and enjoyed it a lot and now it was time to move my skills to a different level making sure enjoying myself as a software tester is as well in scope as the managing bit. I really wanted to get back to what I like most: software testing, setting up a testing process, showing developers how things can be better when continuous testing is going on, in short “finding solutions by executing and not just managing”.
For the last year I have worked as a test consultant at Polteq Test Services. In this year I have touched a range of things in my work I am extremely passionate about, setting up test automation, attempting to help testers improve themselves and the product they work on, helped review a book, used my network to help companies deliver better products by having them tested, helping out writing commercial offers for potential customers of Polteq and probably more I don’t even remember.

So far I have quite enjoyed the variation in the work and quite enjoy being on site at the customers. The one thing I truly miss though is the direct interaction with my colleagues. When working fully in-house there are always steady colleagues, who share your thoughts and worries about the employer, the atmosphere etc. In consultancy however, quite often you do not have your own colleagues on site, you’re mostly working with the customer. So now and again you want to be able to vent frustrations, whether they are about work, traffic or your customer, it is not always easy to do that when on site.

A side effect of working for a company specialized in software testing is, that I am a lot more involved in the “community” and development of the trade. My twitter stream is a lot more active, I have started blogging about my work, I try to stay in touch with communities and groups on LinkedIn and of course on Software Testing Club.

Extra personal effect of me no longer managing people, I am generally a lot more relaxed at home, I have learned to leave my work behind me and not (well, ok, hardly) take it home with me.

Overall looking back to this year I can say that I enjoyed my new position as a consultant. My expectations were quite high to be honest and  I enjoyed it even more. So far it turned out to be beneficial for both my professional and personal life. I used my skills and capabilities in a totally out-of-box way, discovered new talents and potentials and tried out quite some new activities (book, big presentations, creating a whole new concept / theory, etc).  At the moment I consider this a very good step for my career as this kind of job keeps me motivated and inspired.

Standard

Posted by

Posted on

March 16, 2012

Posted under

Test automation

Comments

7 Comments

Automating a legacy system

In the past I have worked a lot with fairly new, web-based applications. So here and there I would run into some “old” desktop applications, but the majority of my professional career has been, working with web-based technologies.

In my current assignment however I have hit a nice new technology, at least new to me: COBOL over TN3270

When I started on this assignment I had one big fear: being forced to use one of the bloated commercial tools out there which claim they support TN3270 off the shelf. Oh, did I say bloated? That sounds like I am biased towards the so called big commercial tools.

This sense of bias is partially true, I am in favor of using small drivers like WebDriver or White where possible over using a full environment such as QTP or Rational Functional Tester. The big tools quite probably have  their own unique selling points, however for a small IT organisation they are a huge investment and quite often overkill compared to what is actually needed.

When we started looking into tools we almost instantly dismissed the “big ones”, not so much due to their cost, but mainly due to the amount of customization still needed to make them work “out of the box” with TN3270 and the rest of the environment we’re automating. When talking to the vendors they all claimed that their tool is excellent to use for an environment for this, when asked however whether the tool will be pluggable out of the box the answer was consistently no, you will need to do some customization (or better yet, one of their consultants would need to do some customization). This same amount of customization will need to be done with a small driver as well, so why spend a fortune on something that is no better than a small, cheaper or even free tool?

For the Proof of Concept we decided to take two different drivers: Jagacy and s3270.

Jagacy is in their own words:

“… our award winning 3270 screen-scraping library written entirely in Java. It supports SSL, TN3270E, and over thirty languages. It also includes a 3270 emulator designed to help create screen-scraping applications. Developers can also develop their own custom terminal emulators (with automated logon and logoff). Jagacy 3270 screen-scraping is faster, easier to use, and more intuitive than HLLAPI. It excels in creating applications reliably and quickly.”

I cannot but agree with especially the latter part: it is indeed faster and easier than HLLAPI. In terms of more intuitive, I am not quite convinced (yet).

s3270 is a lot more simplistic than Jagacy:

” opens a telnet connection to an IBM host, then allows a script to control the host login session. It is derived from x3270(1), an X-windows IBM 3270 emulator. It implements RFCs 2355 (TN3270E), 1576 (TN3270) and 1646 (LU name selection), and supports IND$FILE file transfer.”

In other words, s3270 can be used as a layer between an application and a legacy TN3270 system.

Talking directly to s3270 is not the most intuitive thing, this requires the code to start a process, keep track of the state of this process and throw input into this process stream. All nice and doable, however it was already invented before by others and they quite likely did an adequate job at making this work, considering TN3270 is a fairly old protocol.

In order to make life for us easier while using s3270 or the PoC, we searched around a bit and found a wonderful open source tool h3270. Put simply, h3270 is a program that allows you to communicate with tn3270 hosts from within your browser. This open source tool helped us quickly hook up our test code to s3270. We dit not however need this browser functionality but are thankfully using the object orientated abstractions within h3270 for our own purposes

One of the wonders of working with tn3270 is that a whole new world of states opens up. In a web-browser for example, something is either there and visible or it is hidden from the user. Keyboards are never locked, when opening a new page the browser will tell your driver when it is done loading. With tn3270 it is not all that simple. First of all, there are features like a locked keyboard, it works with function keys that have disappeared from our physical keyboards years ago, the state of a screen is unknown most of the time, etc.

On the other hand, a terminal emulator is still nothing other than a form of sorts, just a bit more difficult to read and write.

Basically one of the things we once more saw proven, is that in test automation the basics are always similar. It is just the way you need to talk to the system that may be different.

Standard

Posted by

Posted on

March 14, 2012

Posted under

Software testing

Comments

Leave a comment

My personal, professional code of ethics as a tester

Inspired by both my own previous post, but mostly by a response to my post on the Software Testing Club by Ainers Galvans I decided to follow his good example and write what I believe to be my professional code of ethics as a tester.

What is a Code of Ethics?

A code of ethics, or ethical code, is, in my view the following:

  • a set of rules set up to understand the difference between ‘right’ and ‘wrong’ and to apply this understanding in my decision-making.

My professional code of ethics

I have tried to keep my professional code of ethics short and understandable. Where I believe more than a one-liner is needed I have tried to keep the explanation as short and clear as possible

  • I am always honest and do not attempt to prettify information or feedback. I expect others to treat me likewise.
  • If I can get a job done in less hours than agreed, I will do so and inform my customer that the amount of time spent was less. If I cannot get a job done in the amount of hours agreed, I will inform my customer of this as soon as I can.
  • I honour (mutual) agreements. Both verbal and written agreements as well as meetings. When I agree to something I will do what I can to honor that agreement. E.g. when I agree to come to a meeting I will do what is needed and possible to be at that meeting.
  • I try to waste as little time as possible. Both my own time and my customers’ time is valuable so if I see a way to speed things up without losing efficiency and quality, I will try to speed things up.
  • I need to see the point and added value in the things I am supposed to do. A task or a job that seems pointless to me and of which the added value remains unclear to me gives me zero motivation, which would reflect on the quality of the execution, therefore it is a must for me to see what the added value of  the task is.
  • I will always try to learn something new, regardless of the situation.

Generally I try not to compromise this set, but it is of course not a black and white set of rules, there are shades of grey as there are in all things in life. These are my professional basics which i personally am trying to stick to. There of course might be quite some additional insights to build up on top of those, which are appropriate for and depending on each customer / situation / task etc.

Standard

Posted by

Posted on

February 10, 2012

Posted under

Software testing

Comments

4 Comments

Work ethics – being professional?

No matter where I have worked I always believed, and still do, that I have a strong and sensible work ethic. I expect to have to work hard in order to learn new things and grow in my role. Sometimes working hard means traveling a lot, sometimes it involves working long hours or weekends without being paid extra for it. Quite often I have asked myself whether it is all worth it, generally speaking it always was.

I enjoy learning new things, new techniques or technologies. Learning means you need to invest in it, do something actively, like making an extra effort on top of your regular day-to-day life.

In return for making an extra effort, I generally get appreciation from my colleagues and customers and gain new insights and knowledge. On top of getting appreciation, I also get credits for wanting to go the extra mile, which results in goodwill from both customers and colleagues and strengthens my reputation as a professional.

Of course I too complain about things. Obviously there are things I would like to see done differently in my work. However I try to take the active approach to introduce the improvements and make life easier for all those involved.

I truly believe that in a professional working environment it is not a lot to ask people who complain – and rightly so – about things they consider wrong or up for improvement to be constructive and try to change these things, look for solutions rather than keep whining. Quite often however, when you ask the complaining people what are they planning to do to make things change, silence is the answer.

I understand that not everyone has the same drive for his work, some people rolled into a profession by accident and are not motivated to grow their, work related, knowledge and just work to make a living. They, hopefully, have something else that gives meaning to what they do and which motivates and inspires them everyday. Personally, I enjoy my job. I like testing, I like finding solutions and moving  software testing to the next level, I enjoy solving testing problems and get satisfaction from deep diving into software testing theories and challenges. I also really enjoy spending time with my family and friends, but that does not make them mutually exclusive in my view. Just as I invest in friendships and my relationship with my wife and family, I invest in my relationship with my employer and my customer.

In my professional life I try to address things I do not like or think could be better (at least in my opinion). This is why it still surprises me that quite often I run into people who are not happy with a lot of things, but when you ask them what do they propose to do about it you get no answer.

Example: as a consultant I do not have a lot of direct contact with my employer, ergo, communications between my employer and myself (and my colleagues) is not always as supple and swift as one (and the company) would like. Since I am generally at a customer site I have to read emails from my work in the evenings and weekend. We use quite some online, or cloud-based, tools to make sure we stay in touch, such as Dropbox and LinkedIn. So on my personal smartphone I make sure I have the LinkedIn app up and running so I can keep in touch with what is happening “back in the office”, even during weekends or when at a customer site where I have no access to anything but the systems we’re working on and with (bank-like institutions usually lock off their network so you also cannot get to the outside, and rightfully so).

To make all of this even easier we now have received a smartphone. Not exactly the one I would have preferred, but hey, who am I to complain about an iPhone? I had to configure the mail on it, install a few apps on it, but I now have a fully mobile miniature office with me. My employer pays for the bandwidth I use on the phone so no added costs to me, just added ways to communicate with the “home front” be it my employer, my colleagues, my assignment or my family and friends.

You’d think wonderful inventions like this make life easier for those who do not regularly read their company email or never participate in a discussion on LinkedIn. However instead it raises even more questions and reservations, such as “am I required to configure email on the phone?”, “why wasn’t this done for me already?”, “do you now expect me to read my email in the evenings?”.

Am I crazy to enjoy my work and wanting to go that extra step, if reading your email indeed should be considered an extra step? I think I am the normal one and those moaning and complaining are the crazies, but once you start noticing that a majority of the people around you are crazy, it makes you reconsider. At least it did for me.

I started to wonder whether it is normal that I feel connected to my work and to my employer, that I like responding to questions, statements or comments posted on forums, that I do read my email at the most improbable times, that I do not mind driving 400 km’s to go to my colleagues on the other side of the country to talk to them about software testing and things that keep my mind occupied.

I still believe I am the normal one, just a normal person part of a minority with strong work ethics.

Standard

Posted by

Posted on

February 9, 2012

Posted under

Test automation

Comments

Leave a comment

Are we afraid of test automation?

After having spent the last 9 months working on a test automation implementation for a team I am not running myself I have started to wonder what are the main success factors for test automation, besides the obvious: achieving the goal you set out in the first place?

One of the main factors that will help test automation be a success in your organisation is to not consider test automation a goal, but use it as a means to achieve a goal. The goal should not be something like “have x% of regression automated”, your goal should be something in which test automation can be a means to achieve this goal, for example help free up time for the testers to focus on important things rather than having to spend most of their time in regression testing.

Another very important thing to keep in mind in implementing test automation is that you need to keep a sharp eye out for trying to automate everything. Technically it may be possible, but it is hardly ever a good idea to want to automate everything. Quite some things require human intervention or human interpretation and cannot be simply made into a Boolean (which is what test automation is in the end, something is either true or false).

Look & feel testing, or validating, should in most cases not be automated for example. This for the simple reason that, despite it being possible, will more often than not raise either false positives or more likely, false negatives. Since these tests often require some form of image recognition and comparison a change of screen resolution or a small CSS change (font difference for example) will make the test fail, resulting in either maintenance or tests being made redundant.

For me however, the main show of success is that the automated tests are actually used and maintained actively.

Having a test automation framework and relying on it to do its job is not good enough. Just like every other software automated tests also want some TLC or at least some attention and maintenance.

Something that is still often seen in test automation is that the tests run but with non-determinate errors, in other words errors that are not really due to the testcases but are also definitely not a bug in the system under test.

If you show some tender love and care to your automation suite, these errors will be spotted, investigated and fixed. More often however these errors will be spotted, someone will “quickly” attempt to fix it and fail, after which the “constantly failing test” will be deemed useless and switched off.
Besides non-determinate errors there is another thing I have seen happen a lot in the past.

Some automation engineers spend a lot of time and efforts building a solid framework with clean and clear reporting. They add a lot of test cases, connect the setup to a continuous integration environment and make sure that they keep all tests working and running.
They then go ahead building more and more tests, adding all kinds of nice new tests and possibilities. What gets forgotten often however, is to involve the rest of the department, they do not show and tell to the (manual?) testers, they do not share with the developers. So the developers use their own unit tests for themselves and if they do some functional testing they do it manually and sloppily. The (manual) testers go about their usual business of testing the software, as they have always done. Not considering that a lot of the test automation suite they can use and abuse to make their life easier. They will spend time on data seeding, manually. They will spend time on look and feel verification on several browsers, manually.

All this time the developers and (manual) testers could have been using the automation framework and the existing tests in it to make their life a lot easier.

While writing this down it starts sounding silly to me and unlikely that it happens, yet I have seen this happen time and time again. What is it that makes developers but especially testers afraid of hesitant to use automated tests?
I love using test automation to make my life easier when doing manual testing, despite having an very strong disliking for writing code.

Test automation should be seen as a tool, it is there to make life a hell of a lot easier. It cannot replace manual testing, but it can take all the repetitiveness and tediousness of manual testing away, it can help you get an idea of what kind of state a system is in before you even start testing, but it can also help you, a lot, in getting the system in all kinds of states!