Scala Repo Server is a more powerful alternative to our static schema repository, and its API is a superset of that repository’s API. At the moment it offers the following additional features:
Please be aware that the Scala Repo Server remains in “beta” - we continue to recommend using S3-based static schema repositories for Iglu in conjunction for all production use cases, including with Snowplow; there are no plans to move Iglu Central over to the Scala Repo Server at this time.
Scala Repo Server can now run on AWS Elastic Beanstalk!
Elastic Beanstalk will automatically configure and manage the EC2 instances needed to run the app. Instructions are available on the Setting up Iglu Server on AWS wiki page.
Once we have a Scala Repo Server up-and-running, we can start to interact with it. If you browse to the HTTP root (
/) of your Scala Repo Server’s API, you will see auto-generated Swagger documentation on all of the available API endpoints.
To start with, we need to create a user who can create schemas for a new vendor prefix:
Now let’s grab a schema that we have available and
POST it to Iglu:
It’s important to note:
isPublic=trueflag ensures that our schema is publically visible - particularly important as Snowplow does not support authenticated repositories yet
The Scala Repo Server should now contain two public JSON Schemas - let’s check:
Good! We can see both of our schemas are now available.
This has been a whirlwind tour through the new capabilities of the Scala Repo Server. I’d recommend setting up an instance, consulting the Swagger documentation at root (
/) and trying out some other commands.
We have added a Vagrant quickstart to the project. If you have VirtualBox and Vagrant installed, you can now easily set up development environment for Iglu Server:
We are making it increasingly easy to work with schemas in Snowplow and Iglu. Originally it was necessary to manually write three files:
Then we started to simplify this process with Schema Guru:
schema-guru schemacan generate a JSON schema from a corpus of JSONs
schema-guru ddlcan automatically generate the JSON Paths file and Redshift DDL for a given schema
Future Iglu releases will make the process still easier. Scala Repo Server will be able to automatically generate the JSON Paths and Redshift DDL files when a schema is uploaded; these files will be served by Scala Repo Server, so it will no longer be necessary to host them in GitHub and/or S3 separately.
A word of warning: the Iglu Server API is still evolving, so future releases are unlikely to be backward compatible with this one. Please continue to use a static repo for all production use cases, such as with Snowplow.
Information on setting up the Scala Repo Server is available on the wiki.
If you have any questions or run into any problems, please raise an issue or get in touch with us through the usual channels.