We are pleased to announce the immediate availability of Snowplow 75 Long-Legged Buzzard. This release lets you warehouse the event streams generated by Urban Airship and SendGrid, and also updates our web-recalculate data model.
The new webhook integrations are as follows:
- Urban Airship - for tracking mobile app-related events from Urban Airship using the new Urban Airship Connect product
- SendGrid - for tracking email-related events delivered by SendGrid via SendGrid webhooks
Here are the sections after the fold:
- Urban Airship Connect support
- SendGrid webhook support
- Updated data model
- Roadmap and contributing
- Documentation and help
1. Urban Airship Connect support
The Urban Airship Connect adapter in lets you track mobile app events delivered by Urban Airship Connect. Using this functionality you can warehouse all of your Urban Airship mobile app and push notification events alongside your existing Snowplow events.
For help setting up Urban Airship Connect support, see the Urban Airship Connect setup wiki page.
For technical details on this adapter, see the Urban Airship Connect adapter wiki page.
2. SendGrid webhook support
The SendGrid webhook adapter lets you track all email-related events emitted by SendGrid through their SendGrid webhooks. It’s great for keeping track of your email campaigns, allowing you to see which emails generate the most interest.
For help setting up SendGrid support, see the SendGrid webhook setup wiki page.
For technical details on this adapter, see the SendGrid webhook adapter wiki page.
3. Updated data model
The updated data model is more efficient (consuming less disk space), and now creates a helpful cookie-ID-to-user-ID map. Thanks Bernardo!
4.1 Upgrading EmrEtlRunner and StorageLoader
The latest version of the EmrEtlRunner and StorageLoader are available from our Bintray here.
In your EmrEtlRunner’s
config.yml file, update your
hadoop_enrich job’s version to 1.5.0, like so:
For a complete example, see our sample
You’ll need to deploy the Redshift tables for any webhooks you plan on ingesting into Snowplow. You can find the Redshift table deployment instructions on the corresponding webhook setup wiki pages:
5. Roadmap and contributing
We welcome any contributions of webhook adapters for other services - for details on getting started, see our wiki page How to integrate a webhook into Snowplow. Please note that contributing will require some experience in Scala at this time.
Similarly, if you would like to sponsor the Snowplow team to build a webhook adapter, do get in touch!
Upcoming Snowplow releases include:
- Release 76 Bird TBC, which will refresh our EmrEtlRunner app, including updating Snowplow to using the EMR 4.x AMI series
- Release 77 Bird TBC, which will bring the Kinesis pipeline up-to-date with the most recent Scala Common Enrich releases. This will also include click redirect support in the Scala Stream Collector
- Release 78 Bird TBC, which will introduce basic event deduplication in Hadoop prior to loading Redshift
Note that these releases are always subject to change between now and the actual release date.
6. Documentation and help
Documentation relating to the new webhook support is available on the wiki:
For more details on this release, please check out the R75 Release Notes on GitHub.