Prior to this release both the realtime and batch enrichment processes would let nonsensical timestamps, such as
22017-11-28 10:01:36, through. However, those events would fail loading into the database of your choice.
With Argentomagus, our realtime enrichment process will now reject those events, which will be routed to the “bad rows” event stream.
This data quality improvement will make its way to the batch pipeline in the next release.
On the security side of things, we have made the cross domain policy of the Scala Stream Collector configurable.
First, what is a Flash cross-domain policy? Quoting the Adobe website:
A cross-domain policy file is an XML document that grants a web client, such as Adobe Flash Player or Adobe Acrobat (though not necessarily limited to these), permission to handle data across domains. When clients request content hosted on a particular source domain and that content make requests directed towards a domain other than its own, the remote domain needs to host a cross-domain policy file that grants access to the source domain, allowing the client to continue the transaction.
To allow a Flash media player hosted on another web server to access content from the Adobe Media Server web server, we require a crossdomain.xml file. A typical use case will be HTTP streaming (VOD or Live) to a Flash Player. The crossdomain.xml file grants a web client the required permission to handle data across multiple domains.
A cross-domain policy file gives the necessary permissions when, for example, you are trying to make a request to a Snowplow collector from a Flash game given that both are running on different hosts.
The Scala Stream Collector embeds what was a very permissive cross-domain policy file, giving permission to any domain and not enforcing HTTPS:
With Release 98, we’re completely removing the
/crossdomain.xml route by default - it will have to be manually re-enabled by adding the following
crossDomain section to the configuration:
Rick Bolkey from OneSpot has contributed a whole suite of improvements to the Scala Stream Collector - much appreciated, Rick.
This new feature lets you scan your redirect for a pattern and replaces it with the
network_userid. This is a powerful tool for performing cookie matching, aka “cookie sync”, for sharing your Snowplow third-party cookie IDs with an ad platform or similar.
As an example, let’s say you’ve enabled this feature by adding the following to your configuration:
And you’re making a redirect request to:
The redirect will point to:
123 is the
In Snowplow R93 Virunum, we introduced cookie bounce. The limitation of this feature was that, when running Scala Stream Collectors behind a load balancer, redirects would lose the original request’s scheme and
http would always be assumed.
Now you can leverage a header specifying the original scheme and use it in your redirect with the following configuration:
Note that for AWS Classic ELB, the original request’s scheme is contained in the
X-Forwarded-Proto header; your load balancer may use a different header.
When using redirects, the Scala Stream Collector would leverage the built-in
Location header provided by Akka-HTTP, the HTTP server library used by the Scala Stream Collector.
However, if this redirect contained a URL as a query parameter, this URL would be partially decoded and would not be resolvable. This has been fixed in Argentomagus.
The real-time applications for R98 Argentomagus are available at the following locations:
Docker images for those new artifacts will follow shortly.
Upcoming Snowplow releases will include:
For more details on this release, please check out the release notes on GitHub.
If you have any questions or run into any problems, please visit our Discourse forum.