Package

PonyBench provides a microbenchmarking framework. It is designed to measure the runtime of synchronous and asynchronous operations.

Example Program

The following is a complete program with multiple trivial benchmarks followed by their output.

use "time"

actor Main is BenchmarkList
  new create(env: Env) =>
    PonyBench(env, this)

  fun tag benchmarks(bench: PonyBench) =>
    bench(_Nothing)
    bench(_Fib(5))
    bench(_Fib(10))
    bench(_Fib(20))
    bench(_Timer(10_000))

class iso _Nothing is MicroBenchmark
  // Benchmark absolutely nothing.
  fun name(): String => "Nothing"

  fun apply() =>
    // prevent compiler from optimizing out this operation
    DoNotOptimise[None](None)
    DoNotOptimise.observe()

class iso _Fib is MicroBenchmark
  // Benchmark non-tail-recursive fibonacci
  let _n: U64

  new iso create(n: U64) =>
    _n = n

  fun name(): String =>
    "_Fib(" + _n.string() + ")"

  fun apply() =>
    DoNotOptimise[U64](_fib(_n))
    DoNotOptimise.observe()

  fun _fib(n: U64): U64 =>
    if n < 2 then 1
    else _fib(n - 1) + _fib(n - 2)
    end

class iso _Timer is AsyncMicroBenchmark
  // Asynchronous benchmark of timer.
  let _ts: Timers = Timers
  let _ns: U64

  new iso create(ns: U64) =>
    _ns = ns

  fun name(): String =>
    "_Timer (" + _ns.string() + " ns)"

  fun apply(c: AsyncBenchContinue) =>
    _ts(Timer(
      object iso is TimerNotify
        fun apply(timer: Timer, count: U64 = 0): Bool =>
          // signal completion of async benchmark iteration when timer fires
          c.complete()
          false
      end,
      _ns))

By default, the results are printed to stdout like so:

Benchmark results will have their mean and median adjusted for overhead.
You may disable this with --noadjust.

Benchmark                                   mean            median   deviation  iterations
Nothing                                     1 ns              1 ns      ±0.87%     3000000
_Fib(5)                                    12 ns             12 ns      ±1.02%     2000000
_Fib(10)                                  185 ns            184 ns      ±1.03%     1000000
_Fib(20)                                23943 ns          23898 ns      ±1.11%       10000
_Timer (10000ns)                        10360 ns          10238 ns      ±3.25%       10000

The --noadjust option outputs results of the overhead measured prior to each benchmark run followed by the unadjusted benchmark result. An example of the output of this program with --noadjust is as follows:

Benchmark                                   mean            median   deviation  iterations
Benchmark Overhead                        604 ns            603 ns      ±0.58%      300000
Nothing                                   553 ns            553 ns      ±0.30%      300000
Benchmark Overhead                        555 ns            555 ns      ±0.51%      300000
_Fib(5)                                   574 ns            574 ns      ±0.43%      300000
Benchmark Overhead                        554 ns            556 ns      ±0.48%      300000
_Fib(10)                                  822 ns            821 ns      ±0.39%      200000
Benchmark Overhead                        554 ns            553 ns      ±0.65%      300000
_Fib(20)                                30470 ns          30304 ns      ±1.55%        5000
Benchmark Overhead                        552 ns            552 ns      ±0.39%      300000
_Timer (10000 ns)                       10780 ns          10800 ns      ±3.60%       10000

It is recommended that a PonyBench program is compiled with the --runtimebc option, if possible, and run with the --ponynoyield option.

Public Types

• actor PonyBench
• type Benchmark
• trait MicroBenchmark
• trait AsyncMicroBenchmark
• interface BenchmarkList
• class BenchConfig
• class OverheadBenchmark
• class AsyncOverheadBenchmark
• class AsyncBenchContinue