We are pleased to announce version 0.2.0 of SQL Runner. SQL Runner is an open source app, written in Go, that makes it easy to execute SQL statements programmatically as part of the Snowplow data pipeline.

To use SQL Runner, you assemble a playbook i.e. a YAML file that lists the different .sql files to be run and the database they are to be run against. It is possible to specify which sequence the files should be run, and to run files in parallel.

Read on after the jump for:

  1. New features
  2. Upgrading
  3. Getting help

For more information on SQL Runner please view the GitHub repository.

1. New features

The main addition is the ability to run SQL against Redshift and Postgres databases which have SSL security enabled. Many thanks to Dennis Waldron from Space Ape Games for contributing this feature!

Update the database targets in your playbooks to use the new ssl parameter:

  - name: "My Postgres database 1"
    type: postgres
    host: localhost
    database: sql_runner_tests_1
    port: 5432
    username: postgres
    ssl: false # SSL disabled by default, or set to true

Other updates in this version:

  • Fixed typo of “queries executed againt targets” (#20)
  • Updated vagrant up to work with latest Peru version (#25)
  • Replaced bitbucket.org/kardianos/osext with github.com/kardianos/osext (#33)

2. Upgrading

A version of SQL Runner for 64-bit Linux is available on Bintray, download it like so:

$ wget http://dl.bintray.com/snowplow/snowplow-generic/sql_runner_0.2.0_linux_amd64.zip

Once downloaded, unzip it:

$ unzip sql_runner_0.2.0_linux_amd64.zip

Run it like so:

$ ./sql-runner
sql-runner version: 0.2.0
Run playbooks of SQL scripts in series and parallel on Redshift and Postgres
  -help=false: Shows this message
  -playbook="": Playbook of SQL scripts to execute
  -sqlroot="PLAYBOOK": Absolute path to SQL scripts. Use PLAYBOOK and BINARY for those respective paths
  -version=false: Shows the program version

Remember to update your database targets in playbooks to use the new ssl parameter.

3. Getting help

For more details on this release, please check out the SQL Runner 0.2.0 release notes on GitHub.

If you have any questions or run into any problems, please raise an issue or get in touch with us through the usual channels.