Since its inception, it has been possible to use
igluctl to check a static schema registry for many kinds of inconsistencies in JSON schemas that cannot be handled by JSON meta schema, such as
minimum value greater than
maximum or a set of string-specific properties in a field with non-string type.
However, when users were presented with error messages from the linter, they often found it hard to identify the source fo the error, because messages produced by
igluctl did not contain any references to the field in the JSON schema, or the line number. Now
igluctl groups problematic issues by their types and adds corresponding JSON Pointers to facilitate localization of problematic property.
Here’s an example output for problematic schema with three discovered issues:
WARNING: Schema [com.apple/notification_event/jsonschema/1-0-0] contains 3 following issues: 1. Following numeric properties are unbounded (add --skip-checks numericMinMax to omit this check): - /properties/notification - /properties/notification/properties/badge 2. Following optional properties don't allow null type (add --skip-checks optionalNull to omit this check): - /properties/notification Use "type: null" to indicate a field as optional for properties subtitle,launchImageName,userInfo,sound,badge,attachments threadId 3. Following string properties provide no clues about maximum length (add --skip-checks stringLength to omit this check): - /properties/notification/properties/userInfo/properties/aps/properties/alert
As well as introducing JSON Pointers, we have also improved the issue descriptions provided to hopefully make it easier to understand what they mean and how to fix them.
The most significant update to Iglu Server in this release is the introduction of new validation semantics. These are exactly the same as those provided in
igluctl: they both use the same Schema DDL library to power them.
Iglu Server exposes a new
This should produce a detailed linting output in JSON format, preserving JSON Pointers already mentioned in
igluctl section and corresponding messages.
In addition, we have improved all output messages by removing the redundant
status property from response’s body and standardizing all dates outputed on ISO 8601.
From infrastructure point of view, we embedded building a docker image into release process, so the
snowplow-docker-registry.bintray.io/snowplow/iglu-server image should be available immediately after release.
Both core libraries, Schema DDL and Iglu Core, have received updates as well.
Among many dependency bumps (such as circe to 0.10.1 and Scala to 2.12.8) Schema DDL got new functionality that supports JSON Pointers in
igluctl along with more type-safe and lossless API for AST generation.
Also as our company-wide effort, we’ve finished the internal migrations from scalaz to cats and from json4s to circe.
The new Iglu Server release can be downloaded from Bintray.
Unzip the compressed file and you can launch the server with the following command:
The latest igluctl, version 0.6.0, can be downloaded from Bintray here.
This new version doesn’t deprecate any behaviors from the previous version, 0.5.0.
Our biggest priority for Iglu R12 is functionality to support our upcoming Redshift table automigration process as per our recently published RFC.
This will include a lot new functionality in core library as well as in Scala Iglu Client.
For more details on this release, as always do check out the release notes and the relevant documentation pages:
If you have any questions or run into any problems, please raise a question in our Discourse forum.