jmeter - replay logs for load testing - Does any tool simulate the actual user time taken between request?


Keywords:jmeter 


Question: 

I need to load test a site with a simulated user load. For this I intend to record the web server logs for a given 10-minute usage of an average user and use this to replay on multiple concurrent threads to simulate a realistic load.

Here's the tools I've looked at and rejected:

Apache benchmark...can program it to hit the server with dumb requests...but can't simulate a user load.

HTTPerf - Can take user logs as input to simulate the average user but only uses the request URLs from the logs...not the request times.

Tsung - Same problem as HTTPerf...I can record a user session... but I still have to program the user "think" time...

JMeter Can take user session logs as input..no idea if it can simulate a user session including the "think" times.

In summary...there's a bunch of tools that will give me "log replay" but they all seem to use the logs just to gather only request URLs to simulate a user session. None of them actually use the timestamps to simulate a real user and their "think" time.

  1. JMeter seems to be the only tool I haven't tried. Will JMeter solve this problem?
  2. Is this easier solved by using sed/awk on my server logs and creating a ruby script to parse the request URLs AND the "think" times and then launching concurrent threads?

Any other suggestions are welcome too.


6 Answers: 

JMeter is capable of injecting constant and random details into your test plan, check out 4.4 Timers and especially 18.6 Timers:

  • Constant Timer

  • Gaussian Random Timer

  • Uniform Random Timer

  • Constant Throughput Timer

  • Synchronizing Timer

  • BeanShell Timer

  • BSF Timer

  • JSR223 Timer

  • Poisson Random Timer

 

Had a similar problem - needed a tool to replay logs from production sever keeping all gaps as is. Also, wanted to manipulate some http headers.

Being sysadmin, first scripted on bash with ab. Wasn't that great speedwise.

Jmeter is great if only it was simple to use.

So, ended up writing own tool to load test by replaying logs. It is written in ruby. Considering that any tool replaying logs spends most of the time in network io wait, not in cpu, ruby GIL shouldn't be a problem. It is multi-threaded, replays logs as is, respecting gaps between log entries, if you have some :)

If you have varnish running, you can also replay in a real time with varnishncsa. You can get on github:

Hope this may help.

 

Another vote for JMeter, it's a good tool for what you need. But, regardless of the tool, a word on your approach: Sadly, it isn't really possible to just point a tool at a webserver log and get a valid load test in return. There just isn't enough data stored in the logs to give you that (not unless all your pages are completely static).

But it's not so hard, the important point is to identify the peak throughput (requests/sec) you want your site to support and use pacing (timers) and users (threads) to build a test that represents this. Using JMeter, the constant throughput timer is particularly useful for this.

Note. It is important to get this peak throughput level correct because otherwise you either end up with a load that is too high (you waste time solving problems that are not problems) or too low (you don't find problems that are real problems) but if you already have access logs showing site usage then, with a little conservative thinking, this should not be a problem.

 

I solved this by using Celery. Celery by its nature will spawn asynchronous workers which is what you need. A task can also take a "countdown" parameter so you can schedule requests.

The task is fairly simple:

import requests

from celery.task import task


@task(max_retries=0, ignore_result=True)
def get_url(url, user_agent):
    headers = {"User-Agent": user_agent}
    try:
        r = requests.get(url, headers=headers)
    except requests.ConnectionError:
        print "Couldn't fetch %s" % url

I used Django's manage command system to parse the logs because I didn't want to bother learning how to talk to Celery in a plain Python environment. Your log format is probably different so adjust as needed. Partial contents of management/commands/my_command.py (excide the quality - I was rushed):

import sys
import csv

import dateutil

from django.core.management.base import BaseCommand, CommandError
from django.utils import timezone


class Command(BaseCommand):

    def handle(self, *args, **options):
        now = timezone.now()
        URL_PREFIX = "http://my.site.com"
        pth = sys.argv[-1]
        fp = open(pth, "r")
        fieldnames=[
            "ip",
            "dc1",
            "cache_result",
            "datetime_a",
            "datetime_b",
            "request_time",
            "upstream_time",
            "dc2",
            "path_raw",
            "status_code",
            "size_bytes",
            "header_url",
            "user_agent"
        ]
        reader = csv.DictReader(fp, fieldnames=fieldnames, delimiter=" ", quotechar='"')
        reader = list(reader)[-1000:]
        fp.close()
        processed = []
        first_diff = None
        for row in reader:
            method, url, b = row["path_raw"].split()
            if method.lower() not in ("get", "head"):
                continue              
            sent_raw = "%s %s:%s:%s" % tuple(row["datetime_a"].lstrip("[").split(":"))\
                + " " + row["datetime_b"].rstrip("]")
            sent = dateutil.parser.parse(sent_raw)
            if first_diff is None:
                first_diff = (now - sent).seconds + 1
            get_url.apply_async(
                (URL_PREFIX + url, row["user_agent"]),
                countdown=first_diff - (now - sent).seconds
            )

Start Celery with manage.py celery worker -B --loglevel=info.

Run the management command with manage.py my_command /path/to/file.log.

 

Maybe gor?

Gor is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data. It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.

Otherwise the answer about grad the bombarder seems great too.

 

Late for the party, but this example seems to be describing exactly what you are looking for, using Jmeter only.

(The trick is to use a BeanShell timer to have the same exact request interval as from your logs)