Jmeter Tips & Tricks – Tip 5

Tip 5 – Logging made easy

Generally speaking it is useful to write your logs to file when running tests. This will help reproduce your results later on, make life easy with comparing results of different tests and will serve as a useful audit log for your load report later on.
The easy way to save the log is to add a filename string in the “Write results to file” box of your listener like this:

ViewResultsTreeWriteToFile

As you can see in this example I generally specify the logname to be at least a bit meaningful, e.g. which Listener did I use and how many threads did I start (the latter is not visible in this sample).

You do however quite regularly run the same amount of threads more than once, so how do you make clear which run this was? Updating the filename very quickly becomes an nuisance. So why not automate that?

I generally use the ${__time()} function. Making the filename something like this:
${__time(yyyyMMdd-HHmmss)}-LISTENERNAME-NUMBEROFTHREADS.jtl

ViewResultsInTableWriteToFile

This kind of logging results in a set of files which is easily sortable on filename and easily readable based on the amount of threads started for this particular test:

20160404-103706-resultsInTable-10threads.jtl

Jmeter Tips & Tricks – tip 4

Tip 4: grabbing a new value for one variable across requests

Imagine the situation, you have this annoying value popping up on every response, which you need in every subsequent request. For example something like a VIEWSTATE, which is quite common and really annoying. If you lose it or pick up the wrong (one request older) version, your request is invalid and your session broken.

So, the simplistic way to grab that thing is on every request add something like the Regular Expression Extractor with a regExp in it grabbing the correct VIEWSTATE. This however ends up being a bit of a mucky business, because you end up with a lot of RegExp Extractors all over the place, resulting in a very complex JMX file.

The tip

regularexpressionextractorPut the Regular Expression Extractor with all stuff in the right places right at the top of your Thread Group. This will ensure that the Extractor is triggered on each and every request sampler, and thus you always have the most up to date (and correct) version of your variable.

 

Jmeter Tips & Tricks: Tip 3

Tip 3: URL Encoding

This tip is a fairly simple one, but I often use it when building scripts.

URL encoding is quite often needed in heavier webapplications on the variables you extract with the Regular Expression extractor or other extractors.

How do you make this variable URL Encoded when not using the standard “Parameters” but you need to use the Body Data instead?

The given variable we are going to encode is something I see rather too often, a ViewState. In order to grab this thing from the page I have the following regular expression:

Reference Name: VIEWSTATE
Regular Expression: id="__VIEWSTATE" value=(.+?)"

So, now we have a variable ${VIEWSTATE} which needs URL Encoding. The way to do this, within the Body Data is as follows:

${__urlencode(${VIEWSTATE})}

So whatever your variable is, just put it in between the brackets in the ${__urlencode()} and you have your encoded variable.

 

Jmeter Tips & Tricks – Tip 2

Tip 2: Semi-random think time from a user

First of all, adding think-time to a Jmeter script can be done with a host different types of timers. The version I am going to propagate here works well for me, I have the feeling I have proper control over the timer this way and I can predict what it does. This does not imply that using a different way of adding think time is wrong in my view. I just happen to like this version.

My think time consists of two pieces: a Test Action with a Uniform Random Timer attached to it:

|__ Test Action > Action: Pause, Current Thread
   |__ Uniform Random Timer > Random Delay Max & Constant Delay Offset set

So, what do the Test Action and the Uniform Random Timer do?

The Test Action sampler is simply a sampler you can use to make a script wait for a specified amount of time.

The Uniform Random Timer however is a bit less simple to explain and understand:

The apache site states the following:

This timer pauses each thread request for a random amount of time, with each time interval having the same probability of occurring. The total delay is the sum of the random value and the offset value.

So what that means is the following.

uniform_random_timer

When setting the Random Delay Maximum, the timer will pick any random value between 0 and 2000. Now add to that the Constant Delay Offset and you have the actual wait (or pause) time JMeter will use in the script in between the requests.

 

So in this example the value will be somewhere between 1 and 3 seconds.

Disadvantage of this method: You will need to set this think time around every HTTP Request you want to have think time.

The functions as described in this post as well as in the previous Jmeter Tip can be found in this JMX file.

Jmeter tips & tricks – Tip 1

Over the years I have taught myself a few simple tips & tricks in Jmeter and I thought it might be nice to share them here. Some are simple, some are a bit more complex but I hope all of them will be useful to reuse for anyone. Where possible I will see if I can add an actual JMX file to download for your ease of use.

Tip 1: Crawling URLs

A URL crawler can be used for several things such as:

  1. verifying all URL’s on the site
  2. mimicking random browse behaviour and thus generating random load on the server

Setting up a crawler involves a few steps. First off you will need a thread group, as you need it with everything. Within that thread group you place an HTTP Request going to your base-page, usually the homepage of the web application.

Then the fun part starts, now we add a While Controller with a simple condition added to it:

LAST - exit loop when last sample in loop fails. If the last sample just before the loop failed, don't enter loop.

You can also make it with an empty Condition, however I prefer using this one so that if your script didn’t already break on an error, it will absolutely break on an error before hitting the loop.

Within the While Controller we now place another HTTP Request, where the path is set to

.*

Underneath the HTTP Request now add an HTML Link Parser.

You now should have a structure similar to this:

TestPlan
  |__ Thread Group
             |__ HTTP Request
             |__ While Controller
                       |__ HTTP Request
                                 |__ HTML Link Parser

The Link Parser does nothing other than extract links from the received HTML.

Next up is adding an If Controller to ensure the link we just grabbed indeed is working and if not, send the crawler back to the base page (e.g. the home page you defined in the first HTTP Request).
The If Controller needs some Conditions in order to do this. My preferred option is to add the following bit of JavaScript:

${__javaScript(!${JMeterThread.last_sample_ok})}

As a child node underneath this If Controller you now need to hang yet another HTTP Request, which sends the user back to your initial homepage. When filling in URL’s etc. please keep in mind that you should just keep life simple and make URL’s either read from a CSV file or simply add them as a global variable in the Test Plan.

The entire plan now should look like this:

TestPlan
  |__ Thread Group
       |__ HTTP Request > Home page
       |__ While Controller > Condition "LAST"
                 |__ HTTP Request > Path ".*"
                           |__ HTML Link Parser
                 |__ If Controller > Condition "JavaScript"
                           |__ HTTP Request > Back to Home page

Please be careful in using this script, this might lead to an unintended Denial Of Service attack if ran uncontrolled and with high load!

A sample JMX of this simple version of a web crawler can be found here.

Reusable components in JMeter – Part 2, Include Controller

In an earlier post I discussed why you might want to use reusable components in Jmeter scripts, in that post I only focused on one possible way of doing that, the Module Controller. For the second part, as promised, I want to discuss the possible uses of the Include Controller.

Why use the Include Controller?

The include controller pretty much does what it states, it allows you to include something external into your test scenario. With controller you can include other JMX files into one single JMX file. So imagine a similar scenario as in the previous post, where you create a JMX specifically for the registration on an application, but then through different channels.

Example: A given application has several ways to communicate to it:

  • Consumer webclient
  • Backoffice webclient
  • Mobile application (communicating via an API)

You can build a set of functional load scenarios for each side of the application, resulting in something like a frontend.jmx, backend.jmx and api.jmx

If you now want to have some full performancetests on the application you can combine these sets with the Include Controller into one big set without making it a huge, unmanageable JMX file.

How does it work?

In the JMX files as mentioned above, instead of a Thread Group you now use a Test Fragment to place your test steps in. If you do happen to use a Thread Group the settings for the thread group from the original JMX will be overridden with the settings from the thread group where the JMX is included.

To avoid confusion, the Test Fragment should be used instead.

TestFragment

Given the existing JMX files you need to use, you create a new project in Jmeter, which will be your master controller. Once you have built the different scenarios you need for your full scale performance test, you consider how these three should work together. Running 3 different JMeter instances, each with its own JMX is an option, however it will make life so much easier if you can actually create a proper flow with the different JMX files.
This Script will be relatively simple and may end up looking something like this:
IncludeControllerYou can now easily control several scripts, from within one or more thread groups within 1 Jmeter instance while still keeping things maintainable and reusable.

 

 

Linux side-step – create a notification sound for your phone with linux and youtube

Both myself and my wife are big fans of the hit TV show “The Big Bang Theory” (I am assuming here that you will know of this series, if not LMGTFY), especially of the scenes where Sheldon Cooper gets drunk and misbehaves in the most fun and childish ways (for example here during a speech). One of the scenes in particular we both found hilarious, this is when Sheldon has dropped String Theory as his main focus and completely lost focus. He gets drunk and makes prank calls to Stephen Hawking. The bit that always cracks us up is the “Who am I” bit of the prank calls, so I decided to see if I could make this into a notification sound for my wives phone.

All of this was executed on my MSI GS-60 Ghost-Pro running Manjaro Linux.

What did I need?

First of all, find the video on Youtube, that was easy of course.

Download the video

Downloading a Youtube movie is, at least on Linux quite easy. There is a wonderful Python based tool available called “youtube-dl” which can be found on GitHub. All you need for that is Python running on your machine.

Follow the basic instructions as explained on the site, this will make sure you have youtube-dl immediately in your command-line path on your machine:

sudo curl https://yt-dl.org/downloads/2015.09.09/youtube-dl -o /usr/local/bin/youtube-dl
sudo chmod a+rx /usr/local/bin/youtube-dl

Now that it is “installed” it is instanty ready for use. I decided to not look long for a different video or higher quality sound, since I am going to make it into a phone notification anyway. This video is high enough quality for that purpose.

Downloading the video from youtube with youtube-dl is child’s play! Youtube-dl has a immense amount of possible options to pass if you really want, you can find them all in the documentation section of the youtube-dl site.

Let’s do it the easy way:

youtube-dl https://www.youtube.com/watch?v=aOQp3FHOPyQ

This will save the file in your current directory in the default formatting based on the video title:

The Big Bang Theory - Drunk Sheldon and Geology feat. Stephen Hawking S07E20 [HD]-aOQp3FHOPyQ.mp4

Pull the sound off the MP4

Now that the video is on your PC somewhere the sound needs to be stripped out of the MP4 video file. To do that ffmpeg is the go-to tool for me! As they themselves state:

FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created. It supports the most obscure ancient formats up to the cutting edge. No matter if they were designed by some standards committee, the community or a corporation.

So ensure you have it running on your linux box by punching in something like this:

ffmpeg --version

That should return something like this:

ffmpeg version 2.8 Copyright (c) 2000-2015 the FFmpeg developers
 built with gcc 5.2.0 (GCC)
 configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab
 libavutil 54. 31.100 / 54. 31.100
 libavcodec 56. 60.100 / 56. 60.100
 libavformat 56. 40.101 / 56. 40.101
 libavdevice 56. 4.100 / 56. 4.100
 libavfilter 5. 40.101 / 5. 40.101
 libavresample 2. 1. 0 / 2. 1. 0
 libswscale 3. 1.101 / 3. 1.101
 libswresample 1. 2.101 / 1. 2.101
 libpostproc 53. 3.100 / 53. 3.100

Now that you have established it is installed, we can pull the soundtrack off the MP4 file:

ffmpeg -i The\ Big\ Bang\ Theory\ -\ Drunk\ Sheldon\ and\ Geology\ feat.\ Stephen\ Hawking\ S07E20\ \[HD\]-aOQp3FHOPyQ.mp4 The\ Big\ Bang\ Theory\ -\ Drunk\ Sheldon\ and\ Geology\ feat.\ Stephen\ Hawking\ S07E20\ \[HD\]-aOQp3FHOPyQ.wav

This will write the soundtrack away to a WAV file. At this point I got fed up with the long filename, so shortened it to sheldon.wav

I was now left with a wav file of 3:47 mins, which is way too long for a notification or a ringtone and definitely didn’t capture the right part I was looking for properly.

Soundfiles editing

For editing the Wav and saving is as the resulting MP3 I used a very nice simple tutorial from xmodulo.com

What it comes down to is install Audacity and use that to find the spot you are looking for. The simple way is to start by looking for the general part of the original WAV file you want to use, zoom in on that time stamp either with the CTRL+1 key combination or with the mouse by selecting the timeslot you want and using the zoom buttons from Audacity.

When you have managed to select the exact piece of the file you can copy that selection and paste it into a new file (e.g. open a new instance of Audacity and paste the sound clip there). Now save that new piece of sound and you have your very own specific, custom ringtone or notification sound just like I made: the slow version and the fast version of a drunk Sheldon Cooper making crazy sounds and mocking Stephen Hawking.

Take these sounds and drop them in the notifications directory of you (android) phone, set them as your default sound for your favorite messaging app.

btw, I noticed later on LinuxJournal covers something very close to this story:)