The letter A styled as Alchemists logo. lchemists
Published February 2, 2019 Updated January 28, 2024
Benchmarks Icon

Benchmarks

4.0.0

Benchmarks is a collection of Ruby micro benchmarks which can be cloned and run locally or used as an information point of reference. The various statistics on Ruby performance captured within this project may or may not surprise you.

Features

  • Uses Benchmark IPS to calculate CPU/speed results.

  • Each script is independently executable.

Requirements

Setup

To install, run:

git clone https://github.com/bkuhlmann/benchmarks.git
cd benchmarks
git checkout 4.0.0
bin/setup

Usage

All benchmark scripts are found within the scripts folder and you can run any benchmark using a relative or absolute file path. Example:

scripts/strings/split

The following is a list of all benchmarks (source + results). Again, you run these locally or study the results provided instead.

scripts/arrays/concatenation

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

a = %w[one two three]
b = %w[four five six]

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "#+" do
    a + b
  end

  benchmark.report "#+=" do
    duplicate = a.dup
    duplicate += b
  end

  benchmark.report "#concat" do
    a.dup.concat b
  end

  benchmark.report "#|" do
    a | b
  end

  benchmark.report "#<< + #flatten" do
    (a.dup << b).flatten
  end

  benchmark.report "splat + #flatten" do
    [a, *b].flatten
  end

  benchmark.report "multi-splat" do
    [*a, *b]
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                  #+   845.131k i/100ms
                 #+=   521.126k i/100ms
             #concat   452.571k i/100ms
                  #|   341.623k i/100ms
      #<< + #flatten   149.466k i/100ms
    splat + #flatten   149.001k i/100ms
         multi-splat   434.644k i/100ms
Calculating -------------------------------------
                  #+     10.475M (± 9.5%) i/s -     52.398M in   5.050961s
                 #+=      5.929M (± 7.8%) i/s -     29.704M in   5.037287s
             #concat      5.003M (± 9.8%) i/s -     24.891M in   5.023014s
                  #|      3.695M (± 8.7%) i/s -     18.448M in   5.032592s
      #<< + #flatten      1.519M (± 8.9%) i/s -      7.623M in   5.058189s
    splat + #flatten      1.531M (± 9.0%) i/s -      7.599M in   5.004145s
         multi-splat      4.701M (± 6.3%) i/s -     23.471M in   5.011116s

Comparison:
                  #+: 10474576.2 i/s
                 #+=:  5929151.0 i/s - 1.77x  slower
             #concat:  5002661.4 i/s - 2.09x  slower
         multi-splat:  4700795.0 i/s - 2.23x  slower
                  #|:  3694672.1 i/s - 2.84x  slower
    splat + #flatten:  1530591.1 i/s - 6.84x  slower
      #<< + #flatten:  1518887.3 i/s - 6.90x  slower

scripts/arrays/search

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

list = %w[one two three four five six seven eight nine ten]
pattern = /t/

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("#grep") { list.grep pattern }
  benchmark.report("#select") { list.select { |value| value.match? pattern } }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
               #grep   136.641k i/100ms
             #select   134.822k i/100ms
Calculating -------------------------------------
               #grep      1.439M (± 7.5%) i/s -      7.242M in   5.061585s
             #select      1.403M (± 7.0%) i/s -      7.011M in   5.021935s

Comparison:
               #grep:  1439047.5 i/s
             #select:  1403449.3 i/s - same-ish: difference falls within error

scripts/closures

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

Example = Class.new do
  def echo_implicit text
    yield
    text
  end

  def echo_implicit_guard text
    yield if block_given?
    text
  end

  def echo_explicit text, &block
    yield block
    text
  end

  def echo_explicit_guard text, &block
    yield block if block
    text
  end
end

block_example = Example.new
lambda_example = -> text { text }
proc_example = proc { |text| text }

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "Block (implicit)" do
    block_example.echo_implicit("hi") { "test" }
  end

  benchmark.report "Block (implicit guard)" do
    block_example.echo_implicit_guard("hi") { "test" }
  end

  benchmark.report "Block (explicit)" do
    block_example.echo_explicit("hi") { "test" }
  end

  benchmark.report "Block (explicit guard)" do
    block_example.echo_explicit_guard("hi") { "test" }
  end

  benchmark.report "Lambda" do
    lambda_example.call "test"
  end

  benchmark.report "Proc" do
    proc_example.call "test"
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
    Block (implicit)     2.399M i/100ms
Block (implicit guard)
                         2.308M i/100ms
    Block (explicit)   450.130k i/100ms
Block (explicit guard)
                       449.734k i/100ms
              Lambda     1.669M i/100ms
                Proc     1.645M i/100ms
Calculating -------------------------------------
    Block (implicit)     46.862M (± 0.1%) i/s -    235.143M in   5.017800s
Block (implicit guard)
                         45.439M (± 0.1%) i/s -    228.508M in   5.028862s
    Block (explicit)      5.056M (± 5.3%) i/s -     25.657M in   5.087058s
Block (explicit guard)
                          5.045M (± 5.4%) i/s -     25.185M in   5.004320s
              Lambda     27.275M (± 0.1%) i/s -    136.832M in   5.016843s
                Proc     27.075M (± 0.1%) i/s -    136.520M in   5.042320s

Comparison:
    Block (implicit): 46861784.2 i/s
Block (implicit guard): 45439260.8 i/s - 1.03x  slower
              Lambda: 27274557.0 i/s - 1.72x  slower
                Proc: 27074831.5 i/s - 1.73x  slower
    Block (explicit):  5056252.9 i/s - 9.27x  slower
Block (explicit guard):  5045448.0 i/s - 9.29x  slower

scripts/constants/lookup

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

CONSTANTS = Hash.new

module Constants
  1_000.times { |index| CONSTANTS["EXAMPLE_#{index}"] = const_set "EXAMPLE_#{index}", index }
end

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("#[]") { CONSTANTS["EXAMPLE_666"] }
  benchmark.report("Module.get (symbol)") { Constants.const_get :EXAMPLE_666 }
  benchmark.report("Module.get (string)") { Constants.const_get "EXAMPLE_666" }
  benchmark.report("Object.get") { Object.const_get "Constants::EXAMPLE_666" }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                 #[]     1.813M i/100ms
 Module.get (symbol)     1.961M i/100ms
 Module.get (string)     1.068M i/100ms
          Object.get   734.137k i/100ms
Calculating -------------------------------------
                 #[]     32.213M (± 1.8%) i/s -    161.350M in   5.010440s
 Module.get (symbol)     32.696M (± 1.4%) i/s -    164.717M in   5.038892s
 Module.get (string)     14.125M (± 2.0%) i/s -     71.568M in   5.068803s
          Object.get      8.708M (± 0.9%) i/s -     44.048M in   5.058943s

Comparison:
 Module.get (symbol): 32695695.0 i/s
                 #[]: 32213332.8 i/s - same-ish: difference falls within error
 Module.get (string): 14124876.2 i/s - 2.31x  slower
          Object.get:  8707737.9 i/s - 3.75x  slower

scripts/delegates

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

require "delegate"
require "forwardable"

module Echo
  def self.call(message) = message
end

class ForwardExample
  def initialize operation
    @operation = operation
  end

  def call(...) = operation.call(...)

  private

  attr_reader :operation
end

class DelegateExample
  extend Forwardable

  delegate %i[call] => :operation

  def initialize operation
    @operation = operation
  end

  private

  attr_reader :operation
end

class SimpleExample < SimpleDelegator
end

class ClassExample < DelegateClass Echo
end

message = "A test."
forward_example = ForwardExample.new Echo
deletate_example = DelegateExample.new Echo
simple_example = SimpleExample.new Echo
class_example = ClassExample.new Echo

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("Forward") { forward_example.call message }
  benchmark.report("Delegate") { deletate_example.call message }
  benchmark.report("Simple Delegator") { simple_example.call message }
  benchmark.report("Delegate Class") { class_example.call message }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
             Forward   870.104k i/100ms
            Delegate   804.894k i/100ms
    Simple Delegator   312.778k i/100ms
      Delegate Class   311.149k i/100ms
Calculating -------------------------------------
             Forward     10.839M (± 6.5%) i/s -     54.817M in   5.076450s
            Delegate      9.873M (± 7.4%) i/s -     49.903M in   5.079844s
    Simple Delegator      3.395M (± 7.8%) i/s -     16.890M in   5.004405s
      Delegate Class      3.389M (± 8.1%) i/s -     17.113M in   5.080799s

Comparison:
             Forward: 10838784.1 i/s
            Delegate:  9872509.4 i/s - same-ish: difference falls within error
    Simple Delegator:  3395064.5 i/s - 3.19x  slower
      Delegate Class:  3389384.7 i/s - 3.20x  slower

scripts/hashes/lookup

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

example = {a: 1, b: 2, c: 3}

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("#[]") { example[:b] }
  benchmark.report("#fetch") { example.fetch :b }
  benchmark.report("#fetch (default)") { example.fetch :b, "default" }
  benchmark.report("#fetch (block)") { example.fetch(:b) { "default" } }
  benchmark.report("#dig") { example.dig :b }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                 #[]     1.648M i/100ms
              #fetch     1.547M i/100ms
    #fetch (default)     1.562M i/100ms
      #fetch (block)     1.547M i/100ms
                #dig     1.553M i/100ms
Calculating -------------------------------------
                 #[]     26.018M (± 3.4%) i/s -    130.197M in   5.009925s
              #fetch     23.729M (± 2.5%) i/s -    119.086M in   5.021739s
    #fetch (default)     23.330M (± 3.4%) i/s -    117.134M in   5.026454s
      #fetch (block)     23.443M (± 2.8%) i/s -    117.567M in   5.018956s
                #dig     24.126M (± 1.2%) i/s -    121.144M in   5.021949s

Comparison:
                 #[]: 26018263.6 i/s
                #dig: 24126491.2 i/s - 1.08x  slower
              #fetch: 23729385.7 i/s - 1.10x  slower
      #fetch (block): 23443014.5 i/s - 1.11x  slower
    #fetch (default): 23329743.0 i/s - 1.12x  slower

scripts/hashes/merge

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

extra = {b: 2}

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("Splat") { {a: 1, **extra} }
  benchmark.report("Merge") { {a: 1}.merge extra }
  benchmark.report("Merge!") { {a: 1}.merge! extra }
  benchmark.report("Dup Merge!") { {a: 1}.dup.merge! extra }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
               Splat   643.576k i/100ms
               Merge   446.875k i/100ms
              Merge!   679.384k i/100ms
          Dup Merge!   365.838k i/100ms
Calculating -------------------------------------
               Splat      7.591M (± 6.1%) i/s -     37.971M in   5.022702s
               Merge      5.217M (± 8.3%) i/s -     26.366M in   5.086572s
              Merge!      8.265M (± 7.6%) i/s -     41.442M in   5.045815s
          Dup Merge!      4.122M (± 8.1%) i/s -     20.487M in   5.004504s

Comparison:
              Merge!:  8264520.5 i/s
               Splat:  7590619.9 i/s - same-ish: difference falls within error
               Merge:  5216873.4 i/s - 1.58x  slower
          Dup Merge!:  4122009.8 i/s - 2.00x  slower

scripts/hashes/reduce

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

numbers = {
  one: 1,
  two: 2,
  three: 3,
  four: 4,
  five: 5,
  six: 6,
  seven: 7,
  eight: 8,
  nine: 9,
  ten: 10
}

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "Reduce" do
    numbers.reduce({}) { |collection, (key, value)| collection.merge! value => key }
  end

  benchmark.report "With Object" do
    numbers.each.with_object({}) { |(key, value), collection| collection[value] = key }
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
              Reduce    33.619k i/100ms
         With Object    69.397k i/100ms
Calculating -------------------------------------
              Reduce    349.372k (± 6.8%) i/s -      1.748M in   5.028721s
         With Object    703.163k (± 7.1%) i/s -      3.539M in   5.057035s

Comparison:
         With Object:   703162.6 i/s
              Reduce:   349371.9 i/s - 2.01x  slower

scripts/loops

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

collection = (1..1_000).to_a
sum = 0

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "for" do
    for number in collection do
      sum += number
    end
  end

  benchmark.report "#each" do
    collection.each { |number| sum += number }
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                 for     4.588k i/100ms
               #each     4.641k i/100ms
Calculating -------------------------------------
                 for     45.152k (± 0.4%) i/s -    229.400k in   5.080724s
               #each     46.210k (± 0.4%) i/s -    232.050k in   5.021756s

Comparison:
               #each:    46209.5 i/s
                 for:    45151.9 i/s - 1.02x  slower

scripts/methods/define_method

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

require "forwardable"

Person = Class.new do
  def initialize first, last
    @first = first
    @last = last
  end

  def full_name
    "#{first} #{last}"
  end

  private

  attr_reader :first, :last
end

Example = Class.new Person do
  extend Forwardable

  define_method :unbound_full_name, Person.instance_method(:full_name)
  delegate %i[full_name] => :person

  def initialize first, last, person: Person.new(first, last)
    super first, last
    @person = person
  end

  def wrapped_full_name
    person.full_name
  end

  private

  attr_reader :first, :last, :person
end

example = Example.new "Jill", "Doe"

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("Wrapped") { example.wrapped_full_name }
  benchmark.report("Defined") { example.unbound_full_name }
  benchmark.report("Delegated") { example.full_name }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
             Wrapped   837.082k i/100ms
             Defined   855.620k i/100ms
           Delegated   515.199k i/100ms
Calculating -------------------------------------
             Wrapped     10.248M (± 7.0%) i/s -     51.062M in   5.004285s
             Defined     10.529M (± 6.6%) i/s -     53.048M in   5.058176s
           Delegated      5.821M (± 2.0%) i/s -     29.366M in   5.046785s

Comparison:
             Defined: 10528678.8 i/s
             Wrapped: 10248218.8 i/s - same-ish: difference falls within error
           Delegated:  5821154.3 i/s - 1.81x  slower

scripts/methods/method_proc

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

Example = Class.new do
  def initialize words
    @words = words
    @first_word = words.first
  end

  def direct_single
    say first_word
  end

  def direct_multiple
    words.each { |word| say word }
  end

  def proc_single
    method(:say).call first_word
  end

  def proc_multiple
    words.each { |word| method(:say).call word }
  end

  def method_to_proc_single
    first_word.then(&method(:say))
  end

  def method_to_proc_multiple
    words.each(&method(:say))
  end

  private

  attr_reader :words, :first_word

  def say phrase
    "You said: #{phrase}."
  end
end

example = Example.new %w[one two three]

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("Direct (s)") { example.direct_single }
  benchmark.report("Direct (m)") { example.direct_multiple }
  benchmark.report("Proc (s)") { example.proc_single }
  benchmark.report("Proc (m)") { example.proc_multiple }
  benchmark.report("Method To Proc (s)") { example.method_to_proc_single }
  benchmark.report("Method To Proc (m)") { example.method_to_proc_multiple }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
          Direct (s)   860.200k i/100ms
          Direct (m)   314.512k i/100ms
            Proc (s)   470.304k i/100ms
            Proc (m)   148.096k i/100ms
  Method To Proc (s)   212.521k i/100ms
  Method To Proc (m)   144.383k i/100ms
Calculating -------------------------------------
          Direct (s)     10.554M (± 6.7%) i/s -     53.332M in   5.073432s
          Direct (m)      3.396M (± 8.4%) i/s -     16.984M in   5.034861s
            Proc (s)      5.246M (± 5.8%) i/s -     26.337M in   5.035817s
            Proc (m)      1.507M (± 4.1%) i/s -      7.553M in   5.021176s
  Method To Proc (s)      2.259M (± 6.2%) i/s -     11.264M in   5.003937s
  Method To Proc (m)      1.549M (± 8.2%) i/s -      7.797M in   5.065793s

Comparison:
          Direct (s): 10554191.6 i/s
            Proc (s):  5245716.2 i/s - 2.01x  slower
          Direct (m):  3396117.4 i/s - 3.11x  slower
  Method To Proc (s):  2258835.4 i/s - 4.67x  slower
  Method To Proc (m):  1548501.1 i/s - 6.82x  slower
            Proc (m):  1506850.9 i/s - 7.00x  slower

scripts/methods/send

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

module Static
  def self.call = rand > 0.5 ? one : two

  def self.one = 1

  def self.two = 2
end

module Dynamic
  def self.with_strings = public_send rand > 0.5 ? "one" : "two"

  def self.with_symbols = public_send rand > 0.5 ? :one : :two

  def self.one = 1

  def self.two = 2
end

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2
  max = 1_000_000

  benchmark.report("Static") { max.times { Static.call } }
  benchmark.report("Dynamic (strings)") { max.times { Dynamic.with_strings } }
  benchmark.report("Dynamic (symbols)") { max.times { Dynamic.with_symbols } }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
              Static     1.000 i/100ms
   Dynamic (strings)     1.000 i/100ms
   Dynamic (symbols)     1.000 i/100ms
Calculating -------------------------------------
              Static     20.047 (± 0.0%) i/s -    101.000 in   5.039175s
   Dynamic (strings)      8.973 (± 0.0%) i/s -     45.000 in   5.015272s
   Dynamic (symbols)     11.698 (± 0.0%) i/s -     59.000 in   5.043676s

Comparison:
              Static:       20.0 i/s
   Dynamic (symbols):       11.7 i/s - 1.71x  slower
   Dynamic (strings):        9.0 i/s - 2.23x  slower

scripts/refinements/import

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

module Import
  def dud = true
end

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "With" do
    Module.new { refine(String) { import_methods Import } }
  end

  benchmark.report "Without" do
    Module.new { def dud = true }
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                With     1.045k i/100ms
             Without   276.674k i/100ms
Calculating -------------------------------------
                With     23.800k (±195.6%) i/s -     29.260k in   5.070785s
             Without      2.779M (± 8.7%) i/s -     13.834M in   5.015788s

Comparison:
             Without:  2778896.1 i/s
                With:    23800.2 i/s - 116.76x  slower

scripts/refinements/initialize

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

module Refines
  refine String do
    def dud = true
  end
end

class With
  using Refines

  def initialize value = "demo"
    @value = value
  end
end

class Without
  def initialize value = "demo"
    @value = value
  end
end

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("With") { With.new }
  benchmark.report("Without") { Without.new }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                With   833.953k i/100ms
             Without   838.454k i/100ms
Calculating -------------------------------------
                With     10.205M (± 7.3%) i/s -     50.871M in   5.008494s
             Without     10.220M (± 7.2%) i/s -     51.146M in   5.027072s

Comparison:
             Without: 10220488.8 i/s
                With: 10205174.2 i/s - same-ish: difference falls within error

scripts/refinements/message

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

module Refines
  refine String do
    def dud = true
  end
end

module With
  using Refines

  def self.call(value) = value.dud
end

module Without
  def self.call(value) = value
end

value = "demo"

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("With") { With.call value }
  benchmark.report("Without") { Without.call value }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                With     1.770M i/100ms
             Without     2.381M i/100ms
Calculating -------------------------------------
                With     29.563M (± 0.1%) i/s -    148.702M in   5.029938s
             Without     49.776M (± 0.1%) i/s -    250.041M in   5.023309s

Comparison:
             Without: 49776136.7 i/s
                With: 29563375.3 i/s - 1.68x  slower

scripts/refinements/refine

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "With" do
    Module.new do
      refine String do
        def dud = true
      end
    end
  end

  benchmark.report "Without" do
    Module.new do
      def dud = true
    end
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                With     1.071k i/100ms
             Without   273.359k i/100ms
Calculating -------------------------------------
                With     23.009k (±190.6%) i/s -     28.917k in   5.110875s
             Without      2.765M (± 8.4%) i/s -     13.941M in   5.077013s

Comparison:
             Without:  2765329.2 i/s
                With:    23009.3 i/s - 120.18x  slower

scripts/strings/concatenation

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

one = "One"
two = "Two"
three = "Three"
four = "Four"
five = "Five"
six = "Six"
seven = "Seven"
eight = "Eight"
nine = "Nine"
ten = "Ten"

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "Implicit (<)" do
    "One" "Two"
  end

  benchmark.report "Implicit (>)" do
    "One" "Two" "Three" "Four" "Five" "Six" "Seven" "Eight" "Nine" "Ten"
  end

  benchmark.report "Interpolation (<)" do
    "#{one} #{two}"
  end

  benchmark.report "Interpolation (>)" do
    "#{one} #{two} #{three} #{four} #{five} #{six} #{seven} #{eight} #{nine} #{ten}"
  end

  benchmark.report "#+ (<)" do
    one + " " + two
  end

  benchmark.report "#+ (>)" do
    one + " " + two + " " + three + " " + four + " " + five + " " + six + " " + seven + " " +
    eight + " " + nine + " " + ten
  end

  # WARNING: Mutation.
  benchmark.report "#concat (<)" do
    one.dup.concat two
  end

  # WARNING: Mutation.
  benchmark.report "#concat (>)" do
    one.dup.concat two, three, four, five, six, seven, eight, nine, ten
  end

  # WARNING: Mutation.
  benchmark.report "#<< (<)" do
    one.dup << two
  end

  # WARNING: Mutation.
  benchmark.report "#<< (>)" do
    one.dup << two << three << four << five << six << seven << eight << nine << ten
  end

  benchmark.report "Array#join (<)" do
    [one, two].join " "
  end

  benchmark.report "Array#join (>)" do
    [one, two, three, four, five, six, seven, eight, nine, ten].join " "
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
        Implicit (<)     2.601M i/100ms
        Implicit (>)     2.640M i/100ms
   Interpolation (<)   898.911k i/100ms
   Interpolation (>)   303.153k i/100ms
              #+ (<)   640.087k i/100ms
              #+ (>)    73.694k i/100ms
         #concat (<)     1.003M i/100ms
         #concat (>)   241.868k i/100ms
             #<< (<)     1.076M i/100ms
             #<< (>)   374.733k i/100ms
      Array#join (<)   557.003k i/100ms
      Array#join (>)   262.493k i/100ms
Calculating -------------------------------------
        Implicit (<)     54.132M (± 2.5%) i/s -    273.116M in   5.048673s
        Implicit (>)     54.714M (± 0.3%) i/s -    274.602M in   5.018882s
   Interpolation (<)     11.071M (± 5.6%) i/s -     55.732M in   5.047673s
   Interpolation (>)      3.267M (± 7.4%) i/s -     16.370M in   5.037346s
              #+ (<)      7.464M (± 7.0%) i/s -     37.765M in   5.086482s
              #+ (>)    742.417k (± 1.1%) i/s -      3.758M in   5.063044s
         #concat (<)     12.788M (± 0.9%) i/s -     64.219M in   5.022186s
         #concat (>)      2.551M (± 7.9%) i/s -     12.819M in   5.056076s
             #<< (<)     13.776M (± 3.9%) i/s -     68.860M in   5.006536s
             #<< (>)      4.105M (± 7.6%) i/s -     20.610M in   5.050318s
      Array#join (<)      6.393M (± 4.5%) i/s -     32.306M in   5.064649s
      Array#join (>)      2.790M (± 9.0%) i/s -     13.912M in   5.025822s

Comparison:
        Implicit (>): 54714264.9 i/s
        Implicit (<): 54131866.0 i/s - same-ish: difference falls within error
             #<< (<): 13775992.5 i/s - 3.97x  slower
         #concat (<): 12788273.5 i/s - 4.28x  slower
   Interpolation (<): 11071433.5 i/s - 4.94x  slower
              #+ (<):  7464136.7 i/s - 7.33x  slower
      Array#join (<):  6392876.9 i/s - 8.56x  slower
             #<< (>):  4105202.5 i/s - 13.33x  slower
   Interpolation (>):  3267084.5 i/s - 16.75x  slower
      Array#join (>):  2790078.5 i/s - 19.61x  slower
         #concat (>):  2551103.4 i/s - 21.45x  slower
              #+ (>):   742417.5 i/s - 73.70x  slower

scripts/strings/matching

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

require "securerandom"

word = SecureRandom.alphanumeric 100
string_matcher = "a"
regex_matcher = /\Aa/

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("#match?") { word.match? regex_matcher }
  benchmark.report("#=~") { word =~ regex_matcher }
  benchmark.report("#start_with? (String)") { word.start_with? string_matcher }
  benchmark.report("#start_with? (Regex)") { word.start_with? regex_matcher }
  benchmark.report("#end_with?") { word.end_with? string_matcher }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
             #match?     1.557M i/100ms
                 #=~   640.701k i/100ms
#start_with? (String)
                         1.805M i/100ms
#start_with? (Regex)   592.021k i/100ms
          #end_with?     1.812M i/100ms
Calculating -------------------------------------
             #match?     23.231M (± 0.5%) i/s -    116.739M in   5.025282s
                 #=~      7.508M (± 0.5%) i/s -     37.801M in   5.035199s
#start_with? (String)
                         28.097M (± 0.4%) i/s -    140.817M in   5.011989s
#start_with? (Regex)      6.710M (± 3.2%) i/s -     33.745M in   5.033979s
          #end_with?     28.779M (± 0.7%) i/s -    144.964M in   5.037311s

Comparison:
          #end_with?: 28779396.2 i/s
#start_with? (String): 28096504.9 i/s - 1.02x  slower
             #match?: 23230938.9 i/s - 1.24x  slower
                 #=~:  7507625.8 i/s - 3.83x  slower
#start_with? (Regex):  6709898.5 i/s - 4.29x  slower

scripts/strings/split

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

require "securerandom"

words = Array.new(100_000) { SecureRandom.alphanumeric 10 }
delimiter = " "
text = words.join delimiter
pattern = /\Aa/

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "Without Block" do
    text.split(delimiter).grep(pattern)
  end

  benchmark.report "With Block" do
    selections = []
    text.split(delimiter) { |word| selections << word if word.match? pattern }
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
       Without Block    11.000 i/100ms
          With Block    11.000 i/100ms
Calculating -------------------------------------
       Without Block    117.849 (± 0.8%) i/s -    594.000 in   5.041144s
          With Block    113.401 (± 0.9%) i/s -    572.000 in   5.044469s

Comparison:
       Without Block:      117.8 i/s
          With Block:      113.4 i/s - 1.04x  slower

scripts/strings/substrings

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

example = "example"

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("#sub (string)") { example.sub "x", "b" }
  benchmark.report("#sub (regex)") { example.sub(/x/, "b") }
  benchmark.report("#gsub (string)") { example.gsub "x", "b" }
  benchmark.report("#gsub (regex)") { example.gsub(/x/, "b") }
  benchmark.report("#tr") { example.tr "x", "b" }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
       #sub (string)   439.628k i/100ms
        #sub (regex)   307.022k i/100ms
      #gsub (string)   362.211k i/100ms
       #gsub (regex)   155.869k i/100ms
                 #tr   747.200k i/100ms
Calculating -------------------------------------
       #sub (string)      4.667M (± 4.8%) i/s -     23.740M in   5.096768s
        #sub (regex)      3.324M (± 8.4%) i/s -     16.579M in   5.022002s
      #gsub (string)      4.024M (± 8.7%) i/s -     20.284M in   5.078637s
       #gsub (regex)      1.593M (± 2.5%) i/s -      8.105M in   5.089611s
                 #tr      8.839M (± 9.2%) i/s -     44.085M in   5.025958s

Comparison:
                 #tr:  8838997.2 i/s
       #sub (string):  4667446.9 i/s - 1.89x  slower
      #gsub (string):  4023935.7 i/s - 2.20x  slower
        #sub (regex):  3323612.6 i/s - 2.66x  slower
       #gsub (regex):  1593427.3 i/s - 5.55x  slower

scripts/thens

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "standard" do
    one, two = "one two".split
    "#{one} + #{two} = #{one + two}"
  end

  benchmark.report "then" do
    "one two".split.then { |one, two| "#{one} + #{two} = #{one + two}" }
  end

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
            standard   287.063k i/100ms
                then   273.661k i/100ms
Calculating -------------------------------------
            standard      3.066M (± 5.4%) i/s -     15.501M in   5.071976s
                then      2.890M (± 4.6%) i/s -     14.504M in   5.029748s

Comparison:
            standard:  3066216.7 i/s
                then:  2890464.2 i/s - same-ish: difference falls within error

scripts/values/inheritance

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

PlotStruct = Struct.new :x, :y

class PlotSubclass < Struct.new :x, :y
end

struct = -> { PlotStruct[x: 1, y: 2] }
subclass = -> { PlotSubclass[x: 1, y: 2] }

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("Struct") { struct.call }
  benchmark.report("Subclass") { subclass.call }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
              Struct   347.644k i/100ms
            Subclass   344.218k i/100ms
Calculating -------------------------------------
              Struct      3.852M (± 6.1%) i/s -     19.468M in   5.072154s
            Subclass      3.732M (± 8.5%) i/s -     18.588M in   5.015045s

Comparison:
              Struct:  3851754.0 i/s
            Subclass:  3732310.1 i/s - same-ish: difference falls within error

scripts/values/initialization

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
  gem "dry-struct"
end

Warning[:performance] = false

require "ostruct"

DataDefault = Data.define :a, :b, :c, :d, :e

DataCustom = Data.define :a, :b, :c, :d, :e do
  def initialize a: 1, b: 2, c: 3, d: 4, e: 5
    super
  end
end

StructDefault = Struct.new :a, :b, :c, :d, :e

StructCustom = Struct.new :a, :b, :c, :d, :e do
  def initialize a: 1, b: 2, c: 3, d: 4, e: 5
    super
  end
end

module Types
  include Dry.Types
end

DryExample = Class.new Dry::Struct do
  attribute :a, Types::Strict::Integer
  attribute :b, Types::Strict::Integer
  attribute :c, Types::Strict::Integer
  attribute :d, Types::Strict::Integer
  attribute :e, Types::Strict::Integer
end

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("Data (positional)") { DataDefault[1, 2, 3, 4, 5] }
  benchmark.report("Data (keyword)") { DataDefault[a: 1, b: 2, c: 3, d: 4, e: 5] }
  benchmark.report("Data (custom)") { DataCustom.new }
  benchmark.report("Struct (positional)") { StructDefault[1, 2, 3, 4, 5] }
  benchmark.report("Struct (keyword)") { StructDefault[a: 1, b: 2, c: 3, d: 4, e: 5] }
  benchmark.report("Struct (custom)") { StructCustom.new }
  benchmark.report("OpenStruct") { OpenStruct.new a: 1, b: 2, c: 3, d: 4, e: 5 }
  benchmark.report("Dry Struct") { DryExample[a: 1, b: 2, c: 3, d: 4, e: 5] }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
   Data (positional)   263.219k i/100ms
      Data (keyword)   268.831k i/100ms
       Data (custom)   180.464k i/100ms
 Struct (positional)   613.904k i/100ms
    Struct (keyword)   253.435k i/100ms
     Struct (custom)   248.250k i/100ms
          OpenStruct   513.000 i/100ms
          Dry Struct    87.480k i/100ms
Calculating -------------------------------------
   Data (positional)      2.752M (±10.4%) i/s -     13.687M in   5.027436s
      Data (keyword)      2.780M (±10.4%) i/s -     13.979M in   5.082351s
       Data (custom)      1.774M (± 2.4%) i/s -      9.023M in   5.088151s
 Struct (positional)      6.830M (± 1.7%) i/s -     34.379M in   5.035416s
    Struct (keyword)      2.662M (±10.2%) i/s -     13.432M in   5.099726s
     Struct (custom)      2.588M (±10.1%) i/s -     12.909M in   5.040115s
          OpenStruct      1.631k (±19.3%) i/s -      8.208k in   5.213960s
          Dry Struct    886.556k (±10.9%) i/s -      4.461M in   5.097822s

Comparison:
 Struct (positional):  6829526.2 i/s
      Data (keyword):  2780207.2 i/s - 2.46x  slower
   Data (positional):  2751887.0 i/s - 2.48x  slower
    Struct (keyword):  2661762.5 i/s - 2.57x  slower
     Struct (custom):  2587550.8 i/s - 2.64x  slower
       Data (custom):  1774425.0 i/s - 3.85x  slower
          Dry Struct:   886556.5 i/s - 7.70x  slower
          OpenStruct:     1630.7 i/s - 4188.06x  slower

ℹ️ `Data` is fastest when members are small (like three or less) but performance degrades when more members are added (like five or more). This is because `Data` always initializes with a `Hash` which is not the case with a `Struct`. Additionally, passing keyword arguments to/from Ruby to Ruby is optimized while to/from Ruby/C is not.

scripts/values/reading

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
  gem "dry-struct"
end

require "ostruct"

DataExample = Data.define :to, :from
StructExample = Struct.new :to, :from

module Types
  include Dry.Types
end

DryExample = Class.new Dry::Struct do
  attribute :to, Types::Strict::String
  attribute :from, Types::Strict::String
end

data = DataExample[to: "Rick", from: "Morty"]
struct = StructExample[to: "Rick", from: "Morty"]
open_struct = OpenStruct.new to: "Rick", from: "Morty"
dry_struct = DryExample[to: "Rick", from: "Morty"]

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("Data") { data.to }
  benchmark.report("Struct") { struct.to }
  benchmark.report("OpenStruct") { open_struct.to }
  benchmark.report("Dry Struct") { dry_struct.to }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                Data     2.543M i/100ms
              Struct     2.558M i/100ms
          OpenStruct     1.538M i/100ms
          Dry Struct     1.571M i/100ms
Calculating -------------------------------------
                Data     52.507M (± 3.1%) i/s -    264.489M in   5.043652s
              Struct     51.870M (± 0.1%) i/s -    260.891M in   5.029669s
          OpenStruct     23.023M (± 1.7%) i/s -    115.327M in   5.010693s
          Dry Struct     24.352M (± 0.9%) i/s -    122.552M in   5.032876s

Comparison:
                Data: 52506933.5 i/s
              Struct: 51870432.9 i/s - same-ish: difference falls within error
          Dry Struct: 24352307.9 i/s - 2.16x  slower
          OpenStruct: 23022934.9 i/s - 2.28x  slower

scripts/values/writing

Source

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

require "ostruct"

DataExample = Data.define :to, :from
StructExample = Struct.new :to, :from

data = DataExample[to: "Rick", from: "Morty"]
struct = StructExample[to: "Rick", from: "Morty"]
open_struct = OpenStruct.new to: "Rick", from: "Morty"

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report("Data") { data.with from: "Summer" }
  benchmark.report("Struct") { struct.from = "Summer" }
  benchmark.report("OpenStruct") { open_struct.from = "Summer" }

  benchmark.compare!
end

Benchmark

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT [arm64-darwin22.6.0]
Warming up --------------------------------------
                Data   213.507k i/100ms
              Struct     2.237M i/100ms
          OpenStruct     1.663M i/100ms
Calculating -------------------------------------
                Data      2.296M (± 6.5%) i/s -     11.529M in   5.044563s
              Struct     42.364M (± 0.1%) i/s -    212.555M in   5.017318s
          OpenStruct     26.183M (± 0.1%) i/s -    131.343M in   5.016276s

Comparison:
              Struct: 42364299.0 i/s
          OpenStruct: 26183309.3 i/s - 1.62x  slower
                Data:  2295988.5 i/s - 18.45x  slower

Development

To contribute, run:

git clone https://github.com/bkuhlmann/benchmarks.git
cd benchmarks
bin/setup

To render documentation for all benchmark scripts, run:

bin/render

This is the same script used to update the documentation within this README.

Tests

To test, run:

bin/rake

Credits