Does any open source I/O benchmarking tool support ramping up IOPS?












0















I know that I can use fio to benchmark my disks with any given static workload. However, does any open source high quality benchmarking tool support doing a test where I select following parameters as constants:




  • Test file size (e.g. 500 MB)

  • Static QD (settable at least to 1, 2, 4 and 8)

  • Workload (e.g. random 4k read over whole file span)

  • Direct I/O access (similar to libaio of fio)

  • Define max latency in µs


And the benchmark should slowly increase IOPS until latency goes over set limit after which the benchmark is done. The test result would be latency for each IOPS value, or better yet, minimum+average+max latency for each IOPS value.



Basically I'm asking a tool that can do similar benchmarking required for this storagereview.com graph:
a graph of latency for each random 4k read over different IOPS values



I know that I can repeatedly run fio with different settings to generate the required data but I'm wondering if there's some premade tool for this purpose. Does such a benchmark tool exists?










share|improve this question





























    0















    I know that I can use fio to benchmark my disks with any given static workload. However, does any open source high quality benchmarking tool support doing a test where I select following parameters as constants:




    • Test file size (e.g. 500 MB)

    • Static QD (settable at least to 1, 2, 4 and 8)

    • Workload (e.g. random 4k read over whole file span)

    • Direct I/O access (similar to libaio of fio)

    • Define max latency in µs


    And the benchmark should slowly increase IOPS until latency goes over set limit after which the benchmark is done. The test result would be latency for each IOPS value, or better yet, minimum+average+max latency for each IOPS value.



    Basically I'm asking a tool that can do similar benchmarking required for this storagereview.com graph:
    a graph of latency for each random 4k read over different IOPS values



    I know that I can repeatedly run fio with different settings to generate the required data but I'm wondering if there's some premade tool for this purpose. Does such a benchmark tool exists?










    share|improve this question



























      0












      0








      0








      I know that I can use fio to benchmark my disks with any given static workload. However, does any open source high quality benchmarking tool support doing a test where I select following parameters as constants:




      • Test file size (e.g. 500 MB)

      • Static QD (settable at least to 1, 2, 4 and 8)

      • Workload (e.g. random 4k read over whole file span)

      • Direct I/O access (similar to libaio of fio)

      • Define max latency in µs


      And the benchmark should slowly increase IOPS until latency goes over set limit after which the benchmark is done. The test result would be latency for each IOPS value, or better yet, minimum+average+max latency for each IOPS value.



      Basically I'm asking a tool that can do similar benchmarking required for this storagereview.com graph:
      a graph of latency for each random 4k read over different IOPS values



      I know that I can repeatedly run fio with different settings to generate the required data but I'm wondering if there's some premade tool for this purpose. Does such a benchmark tool exists?










      share|improve this question
















      I know that I can use fio to benchmark my disks with any given static workload. However, does any open source high quality benchmarking tool support doing a test where I select following parameters as constants:




      • Test file size (e.g. 500 MB)

      • Static QD (settable at least to 1, 2, 4 and 8)

      • Workload (e.g. random 4k read over whole file span)

      • Direct I/O access (similar to libaio of fio)

      • Define max latency in µs


      And the benchmark should slowly increase IOPS until latency goes over set limit after which the benchmark is done. The test result would be latency for each IOPS value, or better yet, minimum+average+max latency for each IOPS value.



      Basically I'm asking a tool that can do similar benchmarking required for this storagereview.com graph:
      a graph of latency for each random 4k read over different IOPS values



      I know that I can repeatedly run fio with different settings to generate the required data but I'm wondering if there's some premade tool for this purpose. Does such a benchmark tool exists?







      performance io benchmarks






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jan 9 at 17:15







      Mikko Rantalainen

















      asked Jan 9 at 16:50









      Mikko RantalainenMikko Rantalainen

      554515




      554515






















          2 Answers
          2






          active

          oldest

          votes


















          1














          fio has an option that discovers the highest IOPS that can be done under a certain latency... From the "I/O latency" section of the fio documentation:




          latency_target=time



          If set, fio will attempt to find the max performance point that the given workload will run at while maintaining a latency below this target. When the unit is omitted, the value is interpreted in microseconds. See latency_window and latency_percentile.




          Please see the whole I/O latency section of the fio help as there are a bunch of operations that interact together. You may also find fio's Steady State mode and the separate tool Diskplorer (which itself drives fio) useful. However, I note you've clarified your question and the aforementioned options/tools don't generate latency numbers at a set number of different "max IOPS" points (however Diskplorer does generate latency / IOPS against I/O depth numbers).



          Away from fio, you could also look at using the vdbench tool that StorageReview themselves actually seem to be using in that review (despite their page claiming that they use fio) but you'll have to wave goodbye to a libaio like submission - I'm fairly sure vdbench doesn't do platform specific AIO because it is trying to be platform agnostic (so it can only use multiple threads/processes to up the depth).






          share|improve this answer


























          • If I've understood correctly, the latency_target allows me to figure one IOPS number for one pre-decided max latency. That does not give me performance numbers for lower IOPS (see the example graph in the question). I can run fio repeatedly with different IOPS numbers and graph the latency but I'm wondering if there exists a better tool for the job.

            – Mikko Rantalainen
            Jan 11 at 12:39











          • I'd say Diskplorer is as close as I've seen in terms of a premade tool that uses fio (and perhaps wouldn't be too much work to change the code do what you wanted). Did you also see my mention of the vdbench tool (which is what Storage Review appear to be using)?

            – Anon
            Jan 12 at 5:49



















          0














          Here's a bash script I cooked up (fio-ramp hosted at github):



          #!/bin/bash
          # Copyright 2019 Mikko Rantalainen
          # License: MIT X License
          #
          # Debian/Ubuntu requirements:
          # sudo apt install fio jq
          #
          # See also: https://fio.readthedocs.io/en/latest/fio_doc.html
          #
          set -e

          if test -z "$1"
          then
          echo "usage: $(basename $0) <result.csv> [fio options]" 1>&2
          echo "<result.csv> will contain CSV with µs latency for different IOPS" 1>&2
          echo " For example, " 1>&2
          echo " $(basename $0) output.csv --blocksize=8k --rw=randwrite --iodepth=4" 1>&2
          echo " will compute IOPS latency values for 8K random write QD4." 1>&2
          # Note: if --numjobs=4 then actual IOPS will be 4x the amount targeted because targeted is per job - prefer increasing iodepth instead!
          fi

          resultfile="$1";
          shift; # remove filename from parameters, left rest for fio

          log10_series()
          {
          count=1
          step=1

          echo 1
          while (( $step < 1000000 ))
          do
          for (( i=1; i < 10; i++ ))
          do
          count=$(( $count + $step ))
          echo $count
          done
          step=$(( 10 * $step ))
          done
          }

          echo "Writing output to '$resultfile' ..."

          # Note: "| while read ..." loop causes shell to create subshell, we have to share data via actual file because variables do not work over subshell boundaries :-/
          best_actual_iops_file=$(mktemp --tmpdir fio-ramp-best-actual-iops.XXXXXXXX.tmp)
          echo 0 > "$best_actual_iops_file"
          trap "rm '$best_actual_iops_file'" EXIT

          echo '"Target IO/s", "Actual IO/s", "Min latency (µs)", "Avg latency (µs)", "Max latency (µs)"' | tee "$resultfile"
          log10_series | while read iops
          do
          LC_ALL=C fio --name TEST --filename=fio-ramp.benchmark.temp --rw=randread
          --size=500m --io_size=10g --blocksize=4k
          --ioengine=libaio --direct=1 --numjobs=1 --iodepth=1
          --ramp_time=1 --runtime=5 --end_fsync=1 --group_reporting
          --rate_iops=$iops --rate_iops_min=1 --max_latency=1s
          --warnings-fatal --output-format=json "$@"
          | jq '.jobs | (.read.iops, .read.lat.min, .read.lat.mean, .read.lat.max)'
          | xargs -r printf "%s %s %s %sn" | while read actual_iops min avg max
          do
          printf "% 13s, % 13s, % 18s, % 18s, % 18sn" "$iops" "$actual_iops" "$min" "$avg" "$max" | tee -a "$resultfile"
          if [ "$(echo "$(cat "$best_actual_iops_file") <= $actual_iops" | bc -l)" == "1" ]; then
          echo "$actual_iops" > "$best_actual_iops_file"
          else
          echo "Actual IOPS dropped when target IOPS was increased, aborting." 1>&2
          exit 1
          fi
          done
          done


          Example graph (Latency vs IOPS on Intel SSD 910, random 4K read QD32, log-log graph):
          Graph displaying lowest average latency around 30,000 IOPS and steep increase in latency above 60,000 IOPS.



          Compared to Diskplorer this runs a new fio process for each IOPS target and collects minimum, average and maximum latency. I know bash better than python so this was easier for me to write. In long run improving Diskplorer could be better in case its license is acceptable (currently no license has been defined for that project).






          share|improve this answer



















          • 1





            Nicely done on getting Diskplorer to select a license (I see you the issue you commented on has been solved)! May I suggest that you accept your own answer since technically you made an open source project that solves your question!

            – Anon
            Jan 21 at 7:31













          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "89"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1108347%2fdoes-any-open-source-i-o-benchmarking-tool-support-ramping-up-iops%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          2 Answers
          2






          active

          oldest

          votes








          2 Answers
          2






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          fio has an option that discovers the highest IOPS that can be done under a certain latency... From the "I/O latency" section of the fio documentation:




          latency_target=time



          If set, fio will attempt to find the max performance point that the given workload will run at while maintaining a latency below this target. When the unit is omitted, the value is interpreted in microseconds. See latency_window and latency_percentile.




          Please see the whole I/O latency section of the fio help as there are a bunch of operations that interact together. You may also find fio's Steady State mode and the separate tool Diskplorer (which itself drives fio) useful. However, I note you've clarified your question and the aforementioned options/tools don't generate latency numbers at a set number of different "max IOPS" points (however Diskplorer does generate latency / IOPS against I/O depth numbers).



          Away from fio, you could also look at using the vdbench tool that StorageReview themselves actually seem to be using in that review (despite their page claiming that they use fio) but you'll have to wave goodbye to a libaio like submission - I'm fairly sure vdbench doesn't do platform specific AIO because it is trying to be platform agnostic (so it can only use multiple threads/processes to up the depth).






          share|improve this answer


























          • If I've understood correctly, the latency_target allows me to figure one IOPS number for one pre-decided max latency. That does not give me performance numbers for lower IOPS (see the example graph in the question). I can run fio repeatedly with different IOPS numbers and graph the latency but I'm wondering if there exists a better tool for the job.

            – Mikko Rantalainen
            Jan 11 at 12:39











          • I'd say Diskplorer is as close as I've seen in terms of a premade tool that uses fio (and perhaps wouldn't be too much work to change the code do what you wanted). Did you also see my mention of the vdbench tool (which is what Storage Review appear to be using)?

            – Anon
            Jan 12 at 5:49
















          1














          fio has an option that discovers the highest IOPS that can be done under a certain latency... From the "I/O latency" section of the fio documentation:




          latency_target=time



          If set, fio will attempt to find the max performance point that the given workload will run at while maintaining a latency below this target. When the unit is omitted, the value is interpreted in microseconds. See latency_window and latency_percentile.




          Please see the whole I/O latency section of the fio help as there are a bunch of operations that interact together. You may also find fio's Steady State mode and the separate tool Diskplorer (which itself drives fio) useful. However, I note you've clarified your question and the aforementioned options/tools don't generate latency numbers at a set number of different "max IOPS" points (however Diskplorer does generate latency / IOPS against I/O depth numbers).



          Away from fio, you could also look at using the vdbench tool that StorageReview themselves actually seem to be using in that review (despite their page claiming that they use fio) but you'll have to wave goodbye to a libaio like submission - I'm fairly sure vdbench doesn't do platform specific AIO because it is trying to be platform agnostic (so it can only use multiple threads/processes to up the depth).






          share|improve this answer


























          • If I've understood correctly, the latency_target allows me to figure one IOPS number for one pre-decided max latency. That does not give me performance numbers for lower IOPS (see the example graph in the question). I can run fio repeatedly with different IOPS numbers and graph the latency but I'm wondering if there exists a better tool for the job.

            – Mikko Rantalainen
            Jan 11 at 12:39











          • I'd say Diskplorer is as close as I've seen in terms of a premade tool that uses fio (and perhaps wouldn't be too much work to change the code do what you wanted). Did you also see my mention of the vdbench tool (which is what Storage Review appear to be using)?

            – Anon
            Jan 12 at 5:49














          1












          1








          1







          fio has an option that discovers the highest IOPS that can be done under a certain latency... From the "I/O latency" section of the fio documentation:




          latency_target=time



          If set, fio will attempt to find the max performance point that the given workload will run at while maintaining a latency below this target. When the unit is omitted, the value is interpreted in microseconds. See latency_window and latency_percentile.




          Please see the whole I/O latency section of the fio help as there are a bunch of operations that interact together. You may also find fio's Steady State mode and the separate tool Diskplorer (which itself drives fio) useful. However, I note you've clarified your question and the aforementioned options/tools don't generate latency numbers at a set number of different "max IOPS" points (however Diskplorer does generate latency / IOPS against I/O depth numbers).



          Away from fio, you could also look at using the vdbench tool that StorageReview themselves actually seem to be using in that review (despite their page claiming that they use fio) but you'll have to wave goodbye to a libaio like submission - I'm fairly sure vdbench doesn't do platform specific AIO because it is trying to be platform agnostic (so it can only use multiple threads/processes to up the depth).






          share|improve this answer















          fio has an option that discovers the highest IOPS that can be done under a certain latency... From the "I/O latency" section of the fio documentation:




          latency_target=time



          If set, fio will attempt to find the max performance point that the given workload will run at while maintaining a latency below this target. When the unit is omitted, the value is interpreted in microseconds. See latency_window and latency_percentile.




          Please see the whole I/O latency section of the fio help as there are a bunch of operations that interact together. You may also find fio's Steady State mode and the separate tool Diskplorer (which itself drives fio) useful. However, I note you've clarified your question and the aforementioned options/tools don't generate latency numbers at a set number of different "max IOPS" points (however Diskplorer does generate latency / IOPS against I/O depth numbers).



          Away from fio, you could also look at using the vdbench tool that StorageReview themselves actually seem to be using in that review (despite their page claiming that they use fio) but you'll have to wave goodbye to a libaio like submission - I'm fairly sure vdbench doesn't do platform specific AIO because it is trying to be platform agnostic (so it can only use multiple threads/processes to up the depth).







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Jan 12 at 5:46

























          answered Jan 10 at 7:08









          AnonAnon

          1876




          1876













          • If I've understood correctly, the latency_target allows me to figure one IOPS number for one pre-decided max latency. That does not give me performance numbers for lower IOPS (see the example graph in the question). I can run fio repeatedly with different IOPS numbers and graph the latency but I'm wondering if there exists a better tool for the job.

            – Mikko Rantalainen
            Jan 11 at 12:39











          • I'd say Diskplorer is as close as I've seen in terms of a premade tool that uses fio (and perhaps wouldn't be too much work to change the code do what you wanted). Did you also see my mention of the vdbench tool (which is what Storage Review appear to be using)?

            – Anon
            Jan 12 at 5:49



















          • If I've understood correctly, the latency_target allows me to figure one IOPS number for one pre-decided max latency. That does not give me performance numbers for lower IOPS (see the example graph in the question). I can run fio repeatedly with different IOPS numbers and graph the latency but I'm wondering if there exists a better tool for the job.

            – Mikko Rantalainen
            Jan 11 at 12:39











          • I'd say Diskplorer is as close as I've seen in terms of a premade tool that uses fio (and perhaps wouldn't be too much work to change the code do what you wanted). Did you also see my mention of the vdbench tool (which is what Storage Review appear to be using)?

            – Anon
            Jan 12 at 5:49

















          If I've understood correctly, the latency_target allows me to figure one IOPS number for one pre-decided max latency. That does not give me performance numbers for lower IOPS (see the example graph in the question). I can run fio repeatedly with different IOPS numbers and graph the latency but I'm wondering if there exists a better tool for the job.

          – Mikko Rantalainen
          Jan 11 at 12:39





          If I've understood correctly, the latency_target allows me to figure one IOPS number for one pre-decided max latency. That does not give me performance numbers for lower IOPS (see the example graph in the question). I can run fio repeatedly with different IOPS numbers and graph the latency but I'm wondering if there exists a better tool for the job.

          – Mikko Rantalainen
          Jan 11 at 12:39













          I'd say Diskplorer is as close as I've seen in terms of a premade tool that uses fio (and perhaps wouldn't be too much work to change the code do what you wanted). Did you also see my mention of the vdbench tool (which is what Storage Review appear to be using)?

          – Anon
          Jan 12 at 5:49





          I'd say Diskplorer is as close as I've seen in terms of a premade tool that uses fio (and perhaps wouldn't be too much work to change the code do what you wanted). Did you also see my mention of the vdbench tool (which is what Storage Review appear to be using)?

          – Anon
          Jan 12 at 5:49













          0














          Here's a bash script I cooked up (fio-ramp hosted at github):



          #!/bin/bash
          # Copyright 2019 Mikko Rantalainen
          # License: MIT X License
          #
          # Debian/Ubuntu requirements:
          # sudo apt install fio jq
          #
          # See also: https://fio.readthedocs.io/en/latest/fio_doc.html
          #
          set -e

          if test -z "$1"
          then
          echo "usage: $(basename $0) <result.csv> [fio options]" 1>&2
          echo "<result.csv> will contain CSV with µs latency for different IOPS" 1>&2
          echo " For example, " 1>&2
          echo " $(basename $0) output.csv --blocksize=8k --rw=randwrite --iodepth=4" 1>&2
          echo " will compute IOPS latency values for 8K random write QD4." 1>&2
          # Note: if --numjobs=4 then actual IOPS will be 4x the amount targeted because targeted is per job - prefer increasing iodepth instead!
          fi

          resultfile="$1";
          shift; # remove filename from parameters, left rest for fio

          log10_series()
          {
          count=1
          step=1

          echo 1
          while (( $step < 1000000 ))
          do
          for (( i=1; i < 10; i++ ))
          do
          count=$(( $count + $step ))
          echo $count
          done
          step=$(( 10 * $step ))
          done
          }

          echo "Writing output to '$resultfile' ..."

          # Note: "| while read ..." loop causes shell to create subshell, we have to share data via actual file because variables do not work over subshell boundaries :-/
          best_actual_iops_file=$(mktemp --tmpdir fio-ramp-best-actual-iops.XXXXXXXX.tmp)
          echo 0 > "$best_actual_iops_file"
          trap "rm '$best_actual_iops_file'" EXIT

          echo '"Target IO/s", "Actual IO/s", "Min latency (µs)", "Avg latency (µs)", "Max latency (µs)"' | tee "$resultfile"
          log10_series | while read iops
          do
          LC_ALL=C fio --name TEST --filename=fio-ramp.benchmark.temp --rw=randread
          --size=500m --io_size=10g --blocksize=4k
          --ioengine=libaio --direct=1 --numjobs=1 --iodepth=1
          --ramp_time=1 --runtime=5 --end_fsync=1 --group_reporting
          --rate_iops=$iops --rate_iops_min=1 --max_latency=1s
          --warnings-fatal --output-format=json "$@"
          | jq '.jobs | (.read.iops, .read.lat.min, .read.lat.mean, .read.lat.max)'
          | xargs -r printf "%s %s %s %sn" | while read actual_iops min avg max
          do
          printf "% 13s, % 13s, % 18s, % 18s, % 18sn" "$iops" "$actual_iops" "$min" "$avg" "$max" | tee -a "$resultfile"
          if [ "$(echo "$(cat "$best_actual_iops_file") <= $actual_iops" | bc -l)" == "1" ]; then
          echo "$actual_iops" > "$best_actual_iops_file"
          else
          echo "Actual IOPS dropped when target IOPS was increased, aborting." 1>&2
          exit 1
          fi
          done
          done


          Example graph (Latency vs IOPS on Intel SSD 910, random 4K read QD32, log-log graph):
          Graph displaying lowest average latency around 30,000 IOPS and steep increase in latency above 60,000 IOPS.



          Compared to Diskplorer this runs a new fio process for each IOPS target and collects minimum, average and maximum latency. I know bash better than python so this was easier for me to write. In long run improving Diskplorer could be better in case its license is acceptable (currently no license has been defined for that project).






          share|improve this answer



















          • 1





            Nicely done on getting Diskplorer to select a license (I see you the issue you commented on has been solved)! May I suggest that you accept your own answer since technically you made an open source project that solves your question!

            – Anon
            Jan 21 at 7:31


















          0














          Here's a bash script I cooked up (fio-ramp hosted at github):



          #!/bin/bash
          # Copyright 2019 Mikko Rantalainen
          # License: MIT X License
          #
          # Debian/Ubuntu requirements:
          # sudo apt install fio jq
          #
          # See also: https://fio.readthedocs.io/en/latest/fio_doc.html
          #
          set -e

          if test -z "$1"
          then
          echo "usage: $(basename $0) <result.csv> [fio options]" 1>&2
          echo "<result.csv> will contain CSV with µs latency for different IOPS" 1>&2
          echo " For example, " 1>&2
          echo " $(basename $0) output.csv --blocksize=8k --rw=randwrite --iodepth=4" 1>&2
          echo " will compute IOPS latency values for 8K random write QD4." 1>&2
          # Note: if --numjobs=4 then actual IOPS will be 4x the amount targeted because targeted is per job - prefer increasing iodepth instead!
          fi

          resultfile="$1";
          shift; # remove filename from parameters, left rest for fio

          log10_series()
          {
          count=1
          step=1

          echo 1
          while (( $step < 1000000 ))
          do
          for (( i=1; i < 10; i++ ))
          do
          count=$(( $count + $step ))
          echo $count
          done
          step=$(( 10 * $step ))
          done
          }

          echo "Writing output to '$resultfile' ..."

          # Note: "| while read ..." loop causes shell to create subshell, we have to share data via actual file because variables do not work over subshell boundaries :-/
          best_actual_iops_file=$(mktemp --tmpdir fio-ramp-best-actual-iops.XXXXXXXX.tmp)
          echo 0 > "$best_actual_iops_file"
          trap "rm '$best_actual_iops_file'" EXIT

          echo '"Target IO/s", "Actual IO/s", "Min latency (µs)", "Avg latency (µs)", "Max latency (µs)"' | tee "$resultfile"
          log10_series | while read iops
          do
          LC_ALL=C fio --name TEST --filename=fio-ramp.benchmark.temp --rw=randread
          --size=500m --io_size=10g --blocksize=4k
          --ioengine=libaio --direct=1 --numjobs=1 --iodepth=1
          --ramp_time=1 --runtime=5 --end_fsync=1 --group_reporting
          --rate_iops=$iops --rate_iops_min=1 --max_latency=1s
          --warnings-fatal --output-format=json "$@"
          | jq '.jobs | (.read.iops, .read.lat.min, .read.lat.mean, .read.lat.max)'
          | xargs -r printf "%s %s %s %sn" | while read actual_iops min avg max
          do
          printf "% 13s, % 13s, % 18s, % 18s, % 18sn" "$iops" "$actual_iops" "$min" "$avg" "$max" | tee -a "$resultfile"
          if [ "$(echo "$(cat "$best_actual_iops_file") <= $actual_iops" | bc -l)" == "1" ]; then
          echo "$actual_iops" > "$best_actual_iops_file"
          else
          echo "Actual IOPS dropped when target IOPS was increased, aborting." 1>&2
          exit 1
          fi
          done
          done


          Example graph (Latency vs IOPS on Intel SSD 910, random 4K read QD32, log-log graph):
          Graph displaying lowest average latency around 30,000 IOPS and steep increase in latency above 60,000 IOPS.



          Compared to Diskplorer this runs a new fio process for each IOPS target and collects minimum, average and maximum latency. I know bash better than python so this was easier for me to write. In long run improving Diskplorer could be better in case its license is acceptable (currently no license has been defined for that project).






          share|improve this answer



















          • 1





            Nicely done on getting Diskplorer to select a license (I see you the issue you commented on has been solved)! May I suggest that you accept your own answer since technically you made an open source project that solves your question!

            – Anon
            Jan 21 at 7:31
















          0












          0








          0







          Here's a bash script I cooked up (fio-ramp hosted at github):



          #!/bin/bash
          # Copyright 2019 Mikko Rantalainen
          # License: MIT X License
          #
          # Debian/Ubuntu requirements:
          # sudo apt install fio jq
          #
          # See also: https://fio.readthedocs.io/en/latest/fio_doc.html
          #
          set -e

          if test -z "$1"
          then
          echo "usage: $(basename $0) <result.csv> [fio options]" 1>&2
          echo "<result.csv> will contain CSV with µs latency for different IOPS" 1>&2
          echo " For example, " 1>&2
          echo " $(basename $0) output.csv --blocksize=8k --rw=randwrite --iodepth=4" 1>&2
          echo " will compute IOPS latency values for 8K random write QD4." 1>&2
          # Note: if --numjobs=4 then actual IOPS will be 4x the amount targeted because targeted is per job - prefer increasing iodepth instead!
          fi

          resultfile="$1";
          shift; # remove filename from parameters, left rest for fio

          log10_series()
          {
          count=1
          step=1

          echo 1
          while (( $step < 1000000 ))
          do
          for (( i=1; i < 10; i++ ))
          do
          count=$(( $count + $step ))
          echo $count
          done
          step=$(( 10 * $step ))
          done
          }

          echo "Writing output to '$resultfile' ..."

          # Note: "| while read ..." loop causes shell to create subshell, we have to share data via actual file because variables do not work over subshell boundaries :-/
          best_actual_iops_file=$(mktemp --tmpdir fio-ramp-best-actual-iops.XXXXXXXX.tmp)
          echo 0 > "$best_actual_iops_file"
          trap "rm '$best_actual_iops_file'" EXIT

          echo '"Target IO/s", "Actual IO/s", "Min latency (µs)", "Avg latency (µs)", "Max latency (µs)"' | tee "$resultfile"
          log10_series | while read iops
          do
          LC_ALL=C fio --name TEST --filename=fio-ramp.benchmark.temp --rw=randread
          --size=500m --io_size=10g --blocksize=4k
          --ioengine=libaio --direct=1 --numjobs=1 --iodepth=1
          --ramp_time=1 --runtime=5 --end_fsync=1 --group_reporting
          --rate_iops=$iops --rate_iops_min=1 --max_latency=1s
          --warnings-fatal --output-format=json "$@"
          | jq '.jobs | (.read.iops, .read.lat.min, .read.lat.mean, .read.lat.max)'
          | xargs -r printf "%s %s %s %sn" | while read actual_iops min avg max
          do
          printf "% 13s, % 13s, % 18s, % 18s, % 18sn" "$iops" "$actual_iops" "$min" "$avg" "$max" | tee -a "$resultfile"
          if [ "$(echo "$(cat "$best_actual_iops_file") <= $actual_iops" | bc -l)" == "1" ]; then
          echo "$actual_iops" > "$best_actual_iops_file"
          else
          echo "Actual IOPS dropped when target IOPS was increased, aborting." 1>&2
          exit 1
          fi
          done
          done


          Example graph (Latency vs IOPS on Intel SSD 910, random 4K read QD32, log-log graph):
          Graph displaying lowest average latency around 30,000 IOPS and steep increase in latency above 60,000 IOPS.



          Compared to Diskplorer this runs a new fio process for each IOPS target and collects minimum, average and maximum latency. I know bash better than python so this was easier for me to write. In long run improving Diskplorer could be better in case its license is acceptable (currently no license has been defined for that project).






          share|improve this answer













          Here's a bash script I cooked up (fio-ramp hosted at github):



          #!/bin/bash
          # Copyright 2019 Mikko Rantalainen
          # License: MIT X License
          #
          # Debian/Ubuntu requirements:
          # sudo apt install fio jq
          #
          # See also: https://fio.readthedocs.io/en/latest/fio_doc.html
          #
          set -e

          if test -z "$1"
          then
          echo "usage: $(basename $0) <result.csv> [fio options]" 1>&2
          echo "<result.csv> will contain CSV with µs latency for different IOPS" 1>&2
          echo " For example, " 1>&2
          echo " $(basename $0) output.csv --blocksize=8k --rw=randwrite --iodepth=4" 1>&2
          echo " will compute IOPS latency values for 8K random write QD4." 1>&2
          # Note: if --numjobs=4 then actual IOPS will be 4x the amount targeted because targeted is per job - prefer increasing iodepth instead!
          fi

          resultfile="$1";
          shift; # remove filename from parameters, left rest for fio

          log10_series()
          {
          count=1
          step=1

          echo 1
          while (( $step < 1000000 ))
          do
          for (( i=1; i < 10; i++ ))
          do
          count=$(( $count + $step ))
          echo $count
          done
          step=$(( 10 * $step ))
          done
          }

          echo "Writing output to '$resultfile' ..."

          # Note: "| while read ..." loop causes shell to create subshell, we have to share data via actual file because variables do not work over subshell boundaries :-/
          best_actual_iops_file=$(mktemp --tmpdir fio-ramp-best-actual-iops.XXXXXXXX.tmp)
          echo 0 > "$best_actual_iops_file"
          trap "rm '$best_actual_iops_file'" EXIT

          echo '"Target IO/s", "Actual IO/s", "Min latency (µs)", "Avg latency (µs)", "Max latency (µs)"' | tee "$resultfile"
          log10_series | while read iops
          do
          LC_ALL=C fio --name TEST --filename=fio-ramp.benchmark.temp --rw=randread
          --size=500m --io_size=10g --blocksize=4k
          --ioengine=libaio --direct=1 --numjobs=1 --iodepth=1
          --ramp_time=1 --runtime=5 --end_fsync=1 --group_reporting
          --rate_iops=$iops --rate_iops_min=1 --max_latency=1s
          --warnings-fatal --output-format=json "$@"
          | jq '.jobs | (.read.iops, .read.lat.min, .read.lat.mean, .read.lat.max)'
          | xargs -r printf "%s %s %s %sn" | while read actual_iops min avg max
          do
          printf "% 13s, % 13s, % 18s, % 18s, % 18sn" "$iops" "$actual_iops" "$min" "$avg" "$max" | tee -a "$resultfile"
          if [ "$(echo "$(cat "$best_actual_iops_file") <= $actual_iops" | bc -l)" == "1" ]; then
          echo "$actual_iops" > "$best_actual_iops_file"
          else
          echo "Actual IOPS dropped when target IOPS was increased, aborting." 1>&2
          exit 1
          fi
          done
          done


          Example graph (Latency vs IOPS on Intel SSD 910, random 4K read QD32, log-log graph):
          Graph displaying lowest average latency around 30,000 IOPS and steep increase in latency above 60,000 IOPS.



          Compared to Diskplorer this runs a new fio process for each IOPS target and collects minimum, average and maximum latency. I know bash better than python so this was easier for me to write. In long run improving Diskplorer could be better in case its license is acceptable (currently no license has been defined for that project).







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Jan 15 at 11:23









          Mikko RantalainenMikko Rantalainen

          554515




          554515








          • 1





            Nicely done on getting Diskplorer to select a license (I see you the issue you commented on has been solved)! May I suggest that you accept your own answer since technically you made an open source project that solves your question!

            – Anon
            Jan 21 at 7:31
















          • 1





            Nicely done on getting Diskplorer to select a license (I see you the issue you commented on has been solved)! May I suggest that you accept your own answer since technically you made an open source project that solves your question!

            – Anon
            Jan 21 at 7:31










          1




          1





          Nicely done on getting Diskplorer to select a license (I see you the issue you commented on has been solved)! May I suggest that you accept your own answer since technically you made an open source project that solves your question!

          – Anon
          Jan 21 at 7:31







          Nicely done on getting Diskplorer to select a license (I see you the issue you commented on has been solved)! May I suggest that you accept your own answer since technically you made an open source project that solves your question!

          – Anon
          Jan 21 at 7:31




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Ask Ubuntu!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1108347%2fdoes-any-open-source-i-o-benchmarking-tool-support-ramping-up-iops%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Human spaceflight

          Can not write log (Is /dev/pts mounted?) - openpty in Ubuntu-on-Windows?

          張江高科駅