Performance testing tools: How to pick by job, language and CI fit

girl using performance testing tools

Performance testing tools split into two groups that answer different questions: protocol-level load tools such as k6, JMeter, Gatling, Locust and Artillery, and browser tools such as Lighthouse, WebPageTest and sitespeed.io. The right pick follows from your team's language, the protocols you must hit, where the test runs in CI, and how the licence scales with load.

Most teams need one tool from each group, not six trials. The choice changes only if you need legacy protocols or vendor-backed support at very large scale.

Which tool fits which team

Match the tool to who will write and review the tests. k6 suits JavaScript teams that gate CI on thresholds, because tests live in the repo as code. Apache JMeter suits QA teams that need JDBC, JMS or LDAP alongside HTTP and prefer a GUI. Gatling suits JVM teams that want code-reviewed scenarios and high concurrency per machine.

Lighthouse audits front-end performance for a single page, and its score does not predict behavior under concurrent load.

We recommend starting with k6 for load tests and Lighthouse for front-end budgets, because both run in CI with explicit pass/fail thresholds.

The pick shifts with skills: Python teams may prefer Locust, and enterprises with legacy protocols may need LoadRunner. If the team lacks those skills, Netguru's staff augmentation adds QA and DevOps engineers who already work with k6, JMeter or Gatling.

What are performance testing tools used for?

Performance testing tools simulate user traffic against an application to find which load level breaks a latency, throughput, or error target. ISO/IEC 25010 files this under performance efficiency (iso25000.com - ISO 25010 Standard, 2023).

Each test type answers one question and fails on one metric:

Test type Question it answers Fails when
Load Does the system hold its targets at expected peak? p95 latency or error rate crosses the agreed threshold
Stress Where does it break, and how does it fail? p99 climbs steeply, errors rise, throughput plateaus
Spike Does it absorb a sudden surge and recover? Errors during the surge, or p99 slow to return to baseline
Soak Does it stay stable over hours? p95 drifts upward, memory or connections grow

Set pass/fail on p95 and p99 latency percentiles, never on the mean. A mean of 200 ms can sit alongside a p99 of several seconds, and the slowest 1% of requests is often where checkout or login timeouts occur. An illustrative gate is p95 under 500 ms with errors under 1% (Gatling blog on detecting performance regressions).

Distributed load generation matters once a single generator saturates its CPU or network, because the results then measure the generator and not the application.

Front-end tools such as Lighthouse and sitespeed.io cover single-page rendering and Core Web Vitals.

Decision table: Performance testing tools by language, protocol and CI fit

Pick k6 or Locust for API load tests in a code-first team, JMeter for multi-protocol coverage, Gatling for JVM shops, and Lighthouse or WebPageTest for browser metrics. Load tools generate traffic and report p95 and p99 latency percentiles.

Browser tools measure what one page costs a real user.

Licences and latest releases were checked against each project's repository in November 2026: k6 2.3.0, Apache JMeter 5.6.3, Gatling 3.16.0, Locust 2.47.0, Artillery 2.0.34, Lighthouse 13.6.0 and sitespeed.io 42.10.0.

Protocol and API load testing:

Tool License Scripting Protocols Distributed CI fit Best fit
k6 AGPL-3.0 JavaScript HTTP, WebSocket, gRPC k6-operator on Kubernetes, or Grafana Cloud k6 CLI thresholds set the exit code Dev-owned API tests with arrival-rate scenarios
Apache JMeter Apache 2.0 GUI, XML plans, Groovy HTTP, JDBC, JMS, LDAP, SMTP, TCP Built-in controller and workers Non-GUI mode, Maven plugin QA teams testing mixed protocols
Gatling Apache 2.0 (open-source edition) Java, Kotlin, Scala, JS/TS HTTP, WebSocket, SSE, gRPC, JMS, MQTT Gatling Enterprise (commercial) Maven, Gradle and sbt plugins JVM teams with code-reviewed simulations
Locust MIT Python HTTP; others via custom clients Built-in master and workers Headless mode Python teams, custom user behavior
Artillery MPL-2.0 YAML plus JS HTTP, WebSocket, Socket.io, gRPC AWS Lambda or Fargate from the CLI CLI, YAML in repo Node teams wanting low setup
OpenText LoadRunner Commercial C, JavaScript, Java Web, SAP, Citrix and many more Controller and load generators Supported, heavier setup Enterprises with legacy protocols

Browser and front-end performance:

Tool License What it measures CI fit Best fit
Lighthouse Apache 2.0 Lab Core Web Vitals and audits for one URL Lighthouse CI Per-pull-request regression checks
WebPageTest Free public instance, paid plans Real-browser waterfalls from chosen locations API and CLI Diagnosing slow pages by region and network
sitespeed.io MIT Repeated runs, trends, budgets Docker image Long-term front-end dashboards

Google's Core Web Vitals "good" thresholds are LCP at or under 2.5 s, INP at or under 200 ms and CLS at or under 0.1, measured at the 75th percentile of page loads (web.dev).

Lighthouse also powers PageSpeed Insights, which reports a URL's performance on mobile and desktop and suggests fixes. We recommend running Lighthouse or sitespeed.io in CI on every pull request, so front-end regressions surface during development. Keep a load tool in a separate pipeline stage.

For mobile, see our guides to mobile app performance testing and Android performance testing tools. For LLM-backed endpoints, see performance testing in AI projects.

Which tools handle protocol and API load testing?

k6, Apache JMeter, Gatling, Locust, and Artillery all handle HTTP API load testing for free, and they differ in scripting language, protocol reach, and the point where each one fails first.

The open-source cores run in CI with no license cost. Paid tiers mostly add managed cloud runners and dashboards. The table lists what breaks first and when we would not pick each tool.

Tool Engine and scripting Where it breaks first When we would not pick it
k6 Go engine, JavaScript tests (k6 repository) Native protocols are HTTP, WebSocket, and gRPC; anything else needs an xk6 extension built into a custom binary. Scripts run in k6's own JS runtime, not Node, so npm packages need bundling. Tests need JDBC or JMS, or you want to reuse Node libraries.
Apache JMeter Java, plans stored as XML (JMX) Each virtual user is a thread, so memory cost forces more load generators as concurrency grows. JMX diffs are unreadable in code review. The team reviews tests in pull requests, or needs high concurrency from few machines.
Gatling JVM engine; Java, Kotlin, Scala, and JavaScript/TypeScript SDKs (Gatling repository) JVM warm-up skews the first seconds of a run. The open-source edition leaves multi-machine orchestration to you. There is no JVM build tooling, or you need managed distributed runs without paying for Gatling Enterprise.
Locust Python (Locust repository) One worker process saturates one core, so you run a worker per core. HTTP is built in; other protocols need custom clients. You need very high request rates from a single machine.
Artillery Node.js, YAML scenarios with JS hooks (Artillery repository) Engines cover HTTP, WebSocket, Socket.IO, and gRPC, with nothing for JDBC or JMS. Branching logic gets awkward in YAML. You need database or message-broker load, or complex scenario logic.

JMeter stays the broadest option. It covers HTTP, JDBC, JMS, LDAP, and more, and a tester without extensive programming knowledge can build plans in the GUI or run them from the CLI. Use the GUI for authoring and non-GUI mode for CI runs.

The workload model affects your p95 and p99 latency percentiles. JMeter's default thread groups and Locust's default users are closed models: when the system slows down, virtual users wait and send fewer requests, which flatters tail latency. k6, Gatling, and Artillery can inject by arrival rate, so we recommend them when the test must reflect real traffic.

Distributed load generation also differs by tool. JMeter uses remote engines, Locust uses leader and worker processes, k6 runs on Kubernetes through its operator, and Artillery can fan out to AWS Lambda or Fargate.

Our view: start with k6 for a new API test suite, Gatling in a JVM shop, and Locust where Python is already the team's language. Choose JMeter when protocol breadth is a hard requirement, and Artillery for Node teams testing WebSocket or Socket.IO traffic.

k6: Strengths and limits

k6 is a strong fit for API load testing in CI because k6 thresholds turn latency and error budgets into a pass/fail exit code, and its arrival-rate executors hold request rate steady when the system under test slows down (Grafana k6 documentation: Options reference, 2025).

export const options = {
  scenarios: {
    api: {
      executor: 'constant-arrival-rate',
      rate: 50,
      timeUnit: '1s',
      duration: '5m',
      preAllocatedVUs: 100,
      maxVUs: 300,
    },
  },
  thresholds: {
    http_req_duration: ['p(95)<400', 'p(99)<1000'],
    http_req_failed: ['rate<0.01'],
  },
};

If p95 exceeds 400 ms, p99 exceeds 1 s, or more than 1% of requests fail, k6 exits with a non-zero code (99 for threshold failures) and the pipeline step fails. The 400 ms, 1 s and 1% values are illustrative; set them from your own SLOs.

The executor choice matters as much as the thresholds. Closed models such as constant-vus or ramping-vus wait for each response, so a slow server lowers the request rate and flatters the percentiles. Arrival-rate executors keep sending, and dropped_iterations tells you when maxVUs was too low.

The limits are protocol reach (HTTP, WebSocket and gRPC are covered, JDBC and JMS are not), a Go-based JavaScript runtime that does not run arbitrary npm packages, and AGPL-3.0 licensing that legal teams sometimes want reviewed.

Distributed load generation in the open-source version means running the k6 Operator on Kubernetes. Grafana Cloud k6 adds managed runners and dashboards on a paid tier.

.NET performance testing tools

For .NET performance testing, drive the service over HTTP with k6 or Apache JMeter, because both are protocol-level tools that don't depend on the language the server runs.

k6 scripts are JavaScript and JMeter plans run on the JVM, so neither needs a .NET SDK or a build change in your application. Locust works the same way if your team already writes Python.

Gatling has no .NET SDK, so a .NET team using it would script in Java, Kotlin, Scala or TypeScript.

Keep profilers in a separate step. A load tool shows that p95 or p99 latency breached its threshold. A profiler such as dotnet-trace, dotnet-counters, or PerfView then shows which allocation, lock, or query caused it. NBomber is a C#-native load tool for teams that want tests in the solution; check its licence terms for commercial use.

Which tools measure browser and front-end performance?

Lighthouse, WebPageTest, and sitespeed.io measure browser and front-end performance: how quickly one real browser renders a page and how it scores on Core Web Vitals. They show what a single visitor experiences, so they sit beside load tools and don't replace them.

Hub and spoke diagram. At the center is front-end performance measurement with Lighthouse CI as the default gate. Around it are WebPageTest, sitespeed.io, the pull request comment or status check, and HTML and JSON reports in S3.

The three tools side by side:

Tool What it does CI fit
Lighthouse Audits a URL on mobile and desktop profiles and suggests fixes Lighthouse CI runs per pull request with assertions on budgets
WebPageTest Runs tests in real browsers from chosen locations, with waterfalls and filmstrips UI for investigation, API and CLI for automation
sitespeed.io Open-source runner that drives browsers and stores metrics over time CLI and Docker image, with dashboards for trends

We recommend Lighthouse CI as the default regression gate, because it runs in minutes on every pull request. Configure several runs per URL and assert on the median, since single Lighthouse runs vary. Post the result as a pull request comment or status check, and upload the full HTML and JSON reports to an S3 bucket so reviewers can open the detail.

Add sitespeed.io when you need trends across releases, or WebPageTest when you need to diagnose a slow page from a specific location or device. Lighthouse reports Total Blocking Time as a lab proxy for Interaction to Next Paint, so confirm responsiveness against field data.

A Lighthouse score is a single-user measurement taken on one machine. A page can score well and still degrade when hundreds of sessions hit the same API, because queueing and database contention appear only in p95 and p99 latency under concurrent load.

Pair the browser tools with a protocol-level load test from the earlier section, and treat the two as answers to different questions. Browser tools catch heavy bundles, unoptimized images, and render-blocking scripts. Load tools catch server-side saturation.

How do JMeter, Gatling and LoadRunner compare?

Apache JMeter, Gatling, and OpenText LoadRunner differ most in scripting model, resource use per load generator, and cost. JMeter and the Gatling core are free open source, while LoadRunner is a licensed commercial suite.

The three side by side:

Apache JMeter Gatling (open source and Gatling Enterprise) OpenText LoadRunner
Scripting GUI test plans saved as XML, run from the CLI Code: Java, Kotlin, Scala, JavaScript or TypeScript VuGen recorder and scripts in C, JavaScript and other languages
Resource use per generator Thread per virtual user; CPU and memory climb with large user counts Asynchronous engine; fewer resources per generator at high concurrency Heavy agents, sized per license and controller setup
Protocols HTTP, JDBC, JMS, LDAP, FTP, SMTP HTTP, WebSocket, SSE, gRPC, JMS, MQTT Broad set including legacy enterprise protocols
Reporting Built-in HTML dashboard HTML reports; live dashboards in Gatling Enterprise Dedicated Analysis component
Cost model Free (Apache License 2.0) Free core; Gatling Enterprise priced by plan Commercial license, virtual-user based

JMeter's CPU and memory utilization can be very high for a large number of users, so teams usually add distributed generators earlier than they would with Gatling.

We recommend Gatling when the team writes code, keeps tests in version control, and wants high concurrency per machine. Choose JMeter when QA staff need a GUI and broad protocol coverage at no license cost. Choose LoadRunner when you test legacy enterprise protocols and need vendor support.

A caveat on cost: Gatling Enterprise and LoadRunner both charge for distributed runs and reporting, so compare them against the engineer time you would spend building that yourself on open source.

Netguru is a services partner, not a tool vendor, so we pick among these tools per project.

How do you run performance regression tests in CI?

Run performance regression tests in CI by giving every pipeline a short load test with hard p95 and p99 latency thresholds, and failing the build when they break.

Cycle diagram of the performance regression loop in CI with five steps in order: run a short load test on each pull request, compare against thresholds and baseline, fail or pass the build, store main-branch results as the baseline, then repeat on the Next change.

k6 thresholds do this natively. The process exits with a non-zero code when a threshold is crossed, so the CI job fails without any result parsing.

export const options = {
  scenarios: {
    api: {
      executor: 'constant-arrival-rate',
      rate: 50, timeUnit: '1s',
      duration: '2m', preAllocatedVUs: 100,
    },
  },
  thresholds: {
    http_req_duration: ['p(95)<300', 'p(99)<800'],
    http_req_failed: ['rate<0.01'],
  },
};

Gatling expresses the same gate as assertions on the simulation, and the run fails when one does not hold.

setUp(scn.injectOpen(constantUsersPerSec(50).during(120)))
  .protocols(httpProtocol)
  .assertions(
    global().responseTime().percentile(95).lt(300),
    global().failedRequests().percent().lt(1.0)
  );

Teams with existing JMeter plans can wrap them in Taurus and declare pass/fail criteria in YAML, which avoids rewriting scripts to get a gate. All threshold values above are illustrative; derive yours from your latency SLO.

Absolute thresholds catch large regressions but miss slow drift. For drift, store the summary JSON from each main-branch run as a CI artifact and fail a pull request when its p95 is worse than that baseline by an agreed margin, for example 10% (an illustrative value).

A cadence we suggest is a one-to-two-minute smoke test on every pull request, a longer run on merge to main, and a soak test overnight. Gate pull requests on p95 and keep p99 gates for the longer runs, where the sample is large enough to be stable.

Shared runners add noise. When the load generator and the app under test share one CI runner, they compete for CPU, and p99 then tracks runner load more than code changes. Deploy the app to an ephemeral environment, watch generator CPU during the run, and rerun once before failing on a p99 breach.

Load tools do not render pages, so pair them with the Lighthouse CI gate described in the browser section.

When do commercial and enterprise platforms pay off?

Commercial platforms pay off when running distributed load generation yourself costs more engineering time than a license, or when the test needs protocols and governance that open source doesn't ship.

Three meters drive the bill: VU-hours (you pay for virtual users multiplied by test duration), load-generator hours (you pay for the machines), and seat or concurrent-VU licenses (you pay for capacity whether or not you run tests).

Open-source runs avoid the license but not the generator cost. CPU and memory climb quickly at large user counts, especially on thread-per-user engines such as Apache JMeter, so high-concurrency runs still consume instance hours and tuning time.

What each paid layer adds over its open-source core:

Platform Open-source core What the paid layer adds Usual meter
Grafana Cloud k6 k6 Managed generators in multiple regions, stored results, team dashboards VU-hours
Gatling Enterprise Gatling Injector orchestration (cloud or self-hosted), run history, trend comparison Plan tier plus generator usage
BlazeMeter Runs JMeter, Gatling, k6, and Locust scripts Managed scaling, shared reporting, test management Tiered plans by concurrency
OpenText LoadRunner None (commercial) Broad enterprise-protocol coverage, mature analysis tooling Concurrent VUs or VU-hours
NeoLoad None (commercial) Low-code design, enterprise governance, packaged-app support VU-based licenses

We recommend paying in three situations:

  1. Generator operations are eating sprints. If someone maintains runner fleets, region coverage, and result storage, a managed layer on the tool you already script in usually costs less than that person's time.
  2. The system under test isn't plain HTTP. SAP, Citrix, and similar packaged applications are where LoadRunner and NeoLoad earn their license.
  3. Audit and access control are requirements. Role-based access, SSO, and retained run history are standard in paid tiers.

A team of developers writing tests in code, with p95 and p99 thresholds gating CI, rarely needs the second or third reason. Price a managed k6 or Gatling tier against your generator costs before looking at a full enterprise suite.

How do you choose a performance testing tool?

Choose a performance testing tool by working down five filters in order: team skills, protocols, scale, CI fit, and budget. Stop at the first filter that leaves one or two candidates.

  1. Team skills. k6 scripts are JavaScript, Locust scripts are Python, Artillery uses YAML with JavaScript hooks, and Apache JMeter saves test plans as XML through a GUI. Pick the format your team will review in pull requests.
  2. Protocols. HTTP, WebSockets, and gRPC are covered by k6. JMeter adds JDBC, JMS, and FTP samplers for systems that are not HTTP-first.
  3. Scale and workload model. Check for native arrival-rate executors, since k6 has them and Locust is user-based. Then check how distributed load generation works: JMeter and Locust ship a controller-worker mode, while k6 needs an operator or cloud service.
  4. CI fit. The tool needs a CLI run with pass/fail thresholds on p95 and p99 latency. We recommend adding Lighthouse or sitespeed.io to the pipeline for front-end regressions.
  5. Budget. Weigh runner maintenance time against cloud pricing.

Avoid starting on The Grinder or Tsung, whose projects show little recent release activity.

FAQ: Performance testing tools

What tools are used for performance testing?

k6, Apache JMeter, Gatling, Locust, and Artillery generate protocol-level load, while Lighthouse, WebPageTest, and sitespeed.io measure browser-side performance. OpenText LoadRunner is the commercial suite most often used for enterprise protocol coverage. Match the tool to the job, since a load generator says little about rendering and a browser tool says little about capacity under concurrency.

Is JMeter a performance testing tool?

Yes, Apache JMeter is an open-source Java application for load and performance testing, covering HTTP, JDBC, JMS, and other protocols. It stores test plans as XML and runs headless in CI. JMeter's thread-per-user model tends to need more memory per virtual user than k6, so size load generators carefully at high concurrency.

What are the four main types of performance testing?

The four types most often listed are load, stress, spike, and soak testing. Load testing checks expected traffic, stress testing pushes past capacity to find the breaking point, spike testing applies sudden surges, and soak testing runs for hours to expose memory leaks. Some sources add volume or scalability testing, so lists vary.

What are KPIs in performance testing?

KPIs in performance testing are the measurable targets a run must meet: p95 and p99 latency percentiles, throughput, error rate, and resource utilization. An illustrative threshold is p95 under 500 ms at a fixed request rate. Averages hide tail latency, so percentiles are the safer pass/fail gate in CI.

What are typical performance tests?

Typical performance tests are a baseline run at expected load, a ramp to peak traffic, a stress run to failure, and a long soak. Each replays scripted user flows such as login and checkout. We recommend running the baseline on every build and saving the heavier runs for pre-release gates.

Get help choosing a performance testing setup

Choosing between k6 and Lighthouse, or running both, depends on whether your bottleneck sits in the API layer or in the browser, and on how much of that your CI pipeline can enforce. Our engineers can review your stack, traffic profile, and release cadence, then recommend tools and a workload model that fit your team, including performance checks for AI projects.

Send your current setup through our project estimate form and we will suggest where to start.

Agnieszka Galeza

I love performance testing and related topics. I’m fascinated by continues testing and automation. As a Senior Qa Engineer in Netguru, I work a lot with junior colleagues. Privately, I’m a mother of two kids, I am a plants and gardening lover.

We're Netguru

At Netguru we specialize in designing, building, shipping and scaling beautiful, usable products with blazing-fast efficiency.

Let's talk business