We’re about to launch our Data Product Platform after listening to our customers and many data practitioners around the world — people who develop and use data products were telling us a few consistent messages:
The data space is getting much more complex.
There are more environments and tools than ever — so keeping up is a struggle even if you have people who would be considered as ‘Data Engineers’. Many smaller, growing companies don’t.
A shortage of trained people who can deal with all this complexity.
Companies who have ‘Data Engineer’ and similar roles report they are overwhelmed, often because they are spending too much time on non “business goal” oriented tasks such as infrastructure. Many companies don’t have the staff, but their users still require data. Either way there’s a massive skills shortage.
Users are getting more demanding
They want more data, faster turnaround times, and in a variety of consumption formats. They want trusted data, and therefore…
Managing data as a product is the goal
but the existing tools are not up to the job, and there’s no end-to-end simple solution.
So our team at RAW Labs have developed a unique technology platform to provide Data Products as a Service: A collaborative DataOps platform as a service for data APIs.
The platform enables you to do much more, with less effort, as we take away complexity, manage infrastructure and allow data to be read from many sources and output in many formats for your users’ needs. The platform fits into existing ecosystems of tooling, adopting a DataOps and data-as-code approach.
Interested? sign up for early access to our beta program and whilst you’re there, win a Swiss Army Knife for your efforts!
Watch Georges Lagardère explain how to use it — with a simple example video using Olympic games data:
And, for those interested in how it works, keep reading….
RAW Data Product Platform
1. Low-code data query
RAW uses a dialect of standard SQL with simple functional language constructs to enable data access, query and manipulation inside a single powerful and easy-to-use tool. Query nested files (JSON, XML, etc.) or log files using the same simple constructs as tables, columns and flat files such as CSV. Web APIs are also directly accessible so that glue-code is not required.
2. Configuration driven APIs
APIs are created by a simple YAML file, where the API’s definition, metadata, security, and compute class are specified. The compute class can be set per API endpoint, allowing certain APIs needing more resources to use a large cluster. RAW supports multiple resource groups.
3. DataOps environment
RAW harnesses the power of your collaborative software code management environment (e.g. Git) so that data delivery can use the same tools as software delivery, using both DataOps and Data as Code approaches means that iterations are much faster and produce better quality outputs, and everything is tracked and versioned accordingly. RAW syncs with Git automatically so that APIs are immediately available including branches for testing purposes.
4. Platform-as-a-Service
The RAW platform is fully serverless and runs inside Cloud-based environments where multiple resource groups are made available for queries. Single and Multiple tenant options are available including the option to host inside an Enterprise’s Cloud account. The queries are fully isolated from each other using containers, and all data can be encrypted both in-motion and at-rest.
5. RAW Data Engine
The RAW engine is the result of years of research and enables scale-out query execution for distributed, heterogenous data and complex data types. Based on unique monoid comprehension calculus, the engine is built with state-of-the-art ‘JIT’ code generation techniques. This creates high-performance code which also uses a smart caching algorithm with multiple types of cache employed.
6. Administration UI and API Catalog
A UI to administer and view the catalog of APIs, connect to Git repositories, as well as review activity performance, spend, monitor logs and maintain users and RAW roles. APIs can be exported via OpenAPI and integrated with API management tools and catalogs.
7. Enterprise security integration
RAW integrates with Enterprise security systems via standards-based OAuth (Auth0) mechanism, supporting security policies and scopes for controlling data and API entitlements. API Access is fully monitored and can integrate with your API management platform.
Wrap Up
This is only just the start. There’s so much more to creating and managing data as a product. Over the next few years we plan to focus on enabling a simpler, easier and better experience to deliver data.
We will be starting beta testing in February 2022. For further information, sign up for early access to our beta program, and win a Swiss Army Knife for your efforts.
Jeremy Posner, VP Product & Solutions, RAW Labs.
Get Started for Free
More Useful Links:
- Have Questions / Comments? Ask away – join our Community discussion!
- Read our Tutorials and Getting Started Docs
- Why not follow us on LinkedIn, Twitter, YouTube
- Like code? head on over to GitHub
- Developer? Join us! we are looking for bright minds – at all levels of seniority, in databases, distributed systems, UI/UX.
