No credit card required
We’re excited to announce that Snapi, the powerful data manipulation language that drives the RAW Platform, is now open source!
For those who are new to Snapi and RAW, we’d like to take a moment to share the reasons behind our enthusiasm and discuss the features that set it apart.
Background: The RAW Platform
RAW is a developer-first platform engineered to facilitate quick, convenient integration, delivery, and sharing of data services as APIs. As a data platform, it’s built to run in the Cloud with a fully scalable infrastructure that’s managed for you. Its use cases span across the data chain, from real-time access, transformation and sharing..
The core features of RAW include:
- Real-time querying of various data sources including files, data lakes, databases and APIs;
- Real-time data integration and transformation;
- API delivery and management for the data product output;
- The ability to share access to APIs with an unlimited number of users or applications.
RAW is built to be DataOps-ready and automation-friendly. The platform offers GitHub integration and a built-in CI/CD infrastructure for testing and hosting data APIs.
RAW’s low-code scripting language is called Snapi, which makes it straightforward to navigate even the most complex data integration scenarios.
The Snapi Advantage
Snapi is more than just a data manipulation language—it’s a high-level programming language that brings a fresh and simplified approach to data manipulation, designed to filter and combine data from many sources in real time. With roots in M language and SQL, it’s a versatile tool tailored to handle the unique challenges in data manipulation.
Complex Data Support: Unlike SQL and M, which are primarily designed to tabular data, Snapi supports complex data structures found in JSON or XML files. It offers a robust set of operations for manipulating these structures, making the otherwise tedious task of data flattening both easy and accurate.
Built-in Connectors: Snapi includes a vast array of ready-to-use connectors supporting various data formats, data lakes, and databases, which simplifies and accelerates the pipeline set-up process.
AI-Friendly Design: Snapi’s high-level, intuitive syntax is ideal for AI Large Language Models (LLMs), which makes it perfect for automating code generation and understanding the generated code’s correctness. In fact, this is what enables our new “Code with AI” feature, which you can learn about here.
Unique Error Handling: Snapi brings a new perspective to error handling, specifically designed for real-time data manipulation tasks. It allows for continuous program execution even in the event of exceptions or inaccessible or malformed data sources, ensuring seamless operation when reading data in real-time.
Understandability: Snapi places a strong emphasis on easy-to-understand code, enabled through a high-level syntax, a wide selection of built-in connectors, and well-defined operations. Even without a graphical query builder—which we’re in the process of developing as well! —each operation in Snapi is intuitive and involves learning minimal syntax.
Open Sourcing Snapi: The Way Forward
So, why are we open sourcing Snapi now? Because we believe it’s not just for the RAW platform—we see it as a tool that can be universally beneficial. Trust is the foundation of our long-term relationships, and by open sourcing Snapi, we’re opening the doors to collaboration, innovation, and growth.
We encourage developers to use Snapi independently, and if you’re inspired, contribute with new features, plugins, and more. While we’re in the process of finalizing the open-source repository, complete with comprehensive documentation and a contributors’ guide, we’re keen to kick-start the dialogue.
To that end, we invite you to connect with us on Discord. Let’s collaborate, innovate, and elevate the world of data manipulation together.
And of course, check our repo in GitHub. We even included a small command-line tool so that you can run Snapi directly on your machine against your own data!