Snowplow R95 Ellora released with ZSTD support

13 November 2017
We are excited to announce the release of Snowplow R95 Ellora. This release is primarily focused on updates to the atomic.events table for Redshift users, with a much-anticipated switch to the ZSTD encoding. This change in column encoding should lead to significant reductions in the disk space used by atomic.events for all of our users who load their Snowplow to Redshift. If you’d like to know more about R95 Ellora, named after an archaeological site...

RDB Loader R28 released

13 November 2017  •  Anton Parkhomenko
This release concentrated around improving security and stability of RDB Loader as well as addressing an important AWS SSL update, previously flagged in this Discourse post. Starting with this release, both RDB Loader and RDB Shredder versions will have same umbrella-release number. In this post, we will cover: SSH tunnels The AWS SSL update Other changes RDB Shredder updates Upgrading Contributing 1. SSH tunnels 1.1 SSH tunnels 101 SSH tunnels are often used as an...

Possession is 9/10 of the Law

30 October 2017  •  Anthony Mandelli
We’re at a point now where data is a sexy word. Big Data, data science, data analytics- the list of emerging data-focused fields, tools, and products continues to grow. This growth is largely thanks to developing collection technology; as collection tools improve, we find ourselves handling vastly improved data and actively seeking out ways to use it. However, when it comes to utilizing data, most organizations are relatively unsophisticated in their methods. The truth is...

Snowplow Scala Tracker released

18 October 2017  •  Anton Parkhomenko
We are delighted to release version 0.4.0 of the Snowplow Scala Tracker, for tracking events from your Scala apps and services. This refreshment release adds Scala 2.12 support, significantly improves performance and completely removes the outdated Akka and Spray dependencies. Read on for: Performance improvements Removal of Spray and Akka dependencies Scala 2.12 Contributing 1. Performance improvements Historically, we have used the Snowplow Scala Tracker primarily for telemetry in our own applications and libraries, with...

How we Snowplow at Snowplow

18 October 2017  •  Anthony Mandelli
In a blog post by co-founder Alex Dean, he said, “Ensuring that the data is high fidelity is essential to ensuring that any operational and strategic decision making that’s made on the basis of that data is sound.” This concept of high fidelity data is a core component of the Snowplow philosophy; storing granular, event-level data ensures that the resulting internal database is full of deep, rich information that can be sliced and diced in...

Snowplow Docker images released

13 October 2017  •  Ben Fradet
We are thrilled to announce the first batch of official Docker images for Snowplow. This first release focuses on laying the foundations for running a Snowplow real-time pipeline in a Docker-containerized environment. As a result, this release includes images for: The Scala Stream Collector Stream Enrich The Snowplow S3 Loader The Snowplow Elasticsearch Loader Bringing Docker support to Snowplow has been a real community effort - huge thanks to Joshua Cox, Tamas Szuromi and last...

Snowplow 94 Hill of Tara released

10 October 2017  •  Ben Fradet
We are pleased to announce the urgent release of Snowplow 94 Hill of Tara, named after the archaeological complex in Ireland. We take data loss extremely seriously at Snowplow - shortly after the Snowplow 93 Virunum release, routine load testing of another component (the Elasticsearch Loader) detected an active data loss scenario for our core Stream Enrich app, introduced in R93. This data loss manifests itself around auto-scaling of the Stream Enrich component and the...

Snowplow 93 Virunum released

03 October 2017  •  Ben Fradet
We are tremendously excited to announce the release of Snowplow 93 Virunum. This release focuses on a much needed refresh of the real-time pipeline components: the Scala Stream Collector as well as Stream Enrich. It also fixes some long-standing annoyances regarding the Scala Stream Collector. If you’d like to know more about R93 Virunum, named after the ancient Roman city in Austria, please read on after the fold: Scala Stream Collector: detecting blocked third-party cookies...

Snowplow S3 Loader 0.6.0 released

14 September 2017  •  Enes Aldemir
We are pleased to release version 0.6.0 of Snowplow S3 Loader, formerly known as Kinesis S3, our project dedicated to storing data, including Snowplow raw and enriched event streams, to Amazon S3. This post will cover: NSQ Support Support for “AT_TIMESTAMP” as initial position Upgrading Contributing 1. NSQ Support This release introduces NSQ as an event source - it is for this reason that we have renamed the project from Kinesis S3. Adding NSQ support...

Elasticsearch Loader 0.10.0 released

12 September 2017  •  Enes Aldemir
We are thrilled to announce version 0.10.0 of the Snowplow Elasticsearch Loader, our application for writing Snowplow enriched events and more to Elasticsearch. In this post, we will cover: NSQ support Support for writing raw JSONs Support for “AT_TIMESTAMP” as initial position Configuration changes Contributing 1. NSQ Support With this release, we are adding support for NSQ as an event source: the loader can now sink Snowplow enriched events from an NSQ topic to Elasticsearch....