Benchmarks website

Currently, showing the results of benchmarks still requires manual intervention. With time, we want to automatize these tasks.

How to add a new test to the benchmarks website

Current steps are:

  1. Create a new test in soledad/testing/tests/benchmarks, and commit.

  2. Push it to ssh://

  3. Wait until benchmarks stage finishes (so results are posted to elasticsearch).

  4. Update kibana visualizations and dashboards:

    # currently, kibana configurations are in the `scripts` repository.
    cd scripts/elastic/
    ./ -url
  5. Update the benchmarks website:

    # currently, website lives in the `puppet` repository.
    cd puppet/modules/site_benchmarks/
    vim generate-config
    git commit -a -m "[benchmarks] update website with new tests"
    git push


The following steps are needed to have the website be updated automatically with all existing stats:

  • Move kibana-related stuff to the puppet repository.
  • Push kibana-related scripts to the server.
  • Have kibana be updated periodically with new tests (either using a cron job and maybe also use the CI to trigger that?)
  • Modify website generation scripts so they run in the server to generate static html (also cron and CI would be great here).
  • Enable nginx ssi.
  • Modify script to generate a simple html, include the generated html in benchmark website.