New Features in the RAW Visual Studio Code Extension
Our previous blogs have mainly concentrated on types of data product you can create with RAW – the options are limitless as RAW can query and combine files, databases and APIs of all kinds, allowing you to build really useful data products quickly and easily.
This blog showcases some new features in our Visual Studio Code Extension, which is where our developer community spend their time building great APIs to expose and share data.
Scratchpad with 3 Views of Data
First up, let’s execute a simple “hello world” statement – we now see three output views in the Data Preview pane below the Scratchpad pane. This example:
["hello", "world"] is a simple collection of strings, and we now have 3 views: a Raw View, a Tree View, and a Table View. Of course they are equivalent, but sometimes it’s just easier to use one view vs. the other. For instance, you can copy text easier in the Raw View, the Tree View allows copying of sub-nodes easily, and the Table view is better for viewing flat (or flat-ish) data.
By the way, the Output pane also shows the execution time:
Returning complex structures
Query something that returns complex data, such as this RSS feed from CNN, and the different Data Preview panes are more interesting:
RAW Query Properties pane
Look to the bottom left of the VS Code screen, you will now see a RAW Query Properties pane. In here there is the metadata for the structure that’s just been returned in the latest query:
Creating a Typealias
You can copy this structure which is very useful when you want to create a typealias.
For instance, I’ve just clicked ‘copy’ above, and pasted it below, defined a
typealias t and then used it in the equivalent query below:
After this, it’s very easy to tailor the typealias and return just the fields we want, or rearranging the query to return a different structure.
Running a “Describe” on a data source
We can run a describe command on any data source – this returns metadata about the structure in a similar way.
Perform a “Ctrl+Click” (on PC) or “Cmd+Click” (on Mac) on a source (see below)
Doing that in this example brings up two sub-panes in the Data Preview, the left-hand sub-pane is a Data Description, showing format, encoding, confidence, and other metadata. The right-hand sub-pane is the RQL type of the source, which again can be copied to the clipboard as above
Querying large amounts of data
If you have larger amounts of data, for instance the 15 Million rows of NYC Parking Violations, our system is smart with the results. We perform pagination (see the pages at the bottom right corner) and bring the pages into memory as and when you need them (i.e. when you click):
In the Tree View, we split collections into 100 rows at a time. You can then open up any sub-section and it brings those into memory space on demand too:
Lastly, in the Raw view there is a loading of data when you scroll down to 90% of the current screen, this means again that performance is better and memory is minimised in this view too.
Cancelling a long-running query
Finally, you can kill a long-running query under the spinning ‘Executing Query’ wheel:
That’s it for this blog post – We hope you understand how to use some of these features better – they help to make RAW better to query and show large and complex datasets, and use the return types too. Any suggestions are welcome – please contact us if you have ideas, or share on the community
Jeremy Posner, VP Product & Solutions, RAW Labs.
Want to learn more?
- Take a tour of our Demo Catalogue
- Have Questions / Comments? Ask away – join our Community discussion!
- Read our Tutorials and Getting Started Docs
- Why not follow us on LinkedIn, Twitter, YouTube
- Like code? head on over to GitHub
- Developer? Join us! we are looking for bright minds – at all levels of seniority, in databases, distributed systems, UI/UX.
Walkthrough of Major Features, using a worked example to show both Data Consumer and Data Producer perspectives. Code accompanying this demo can be found on GitHub. You can also See Our Demo Catalog Here, where you can register for your...
Powerful new SEO & SEM Data Products with RAW
Create powerful SEO/SEM data products with RAW. This example compares historical search volumes for two keywords / brands over a time period for trend analysis.
Defining and Running Tests for API endpoints
We show how easy it is to create an API test suite using just YAML for your RAW APIs inside VS Code using our RAW Extension: Define tests, execute them and fix your code.