BLOG

New Features in the RAW Visual Studio Code Extension

August 16, 2022
   
Solution
Posted by Jeremy Posner

Intro

Our previous blogs have mainly concentrated on types of data product you can create with RAW – the options are limitless as RAW can query and combine files, databases and APIs of all kinds, allowing you to build really useful data products quickly and easily.

This blog showcases some new features in our Visual Studio Code Extension, which is where our developer community spend their time building great APIs to expose and share data.

After installing the extension, and registering for a free account, the first thing most people do is visit the scratchpad to run a query:


Scratchpad with 3 Views of Data

First up, let’s execute a simple “hello world” statement – we now see three output views in the Data Preview pane below the Scratchpad pane. This example: ["hello", "world"] is a simple collection of strings, and we now have 3 views: a Raw View, a Tree View, and a Table View. Of course they are equivalent, but sometimes it’s just easier to use one view vs. the other. For instance, you can copy text easier in the Raw View, the Tree View allows copying of sub-nodes easily, and the Table view is better for viewing flat (or flat-ish) data.

3 views in the Data Preview

By the way, the Output pane also shows the execution time:

Output execution time.

Returning complex structures

Query something that returns complex data, such as this RSS feed from CNN, and the different Data Preview panes are more interesting:

complex data in Raw View
complex data in Tree view
complex data in Table view

RAW Query Properties pane

Look to the bottom left of the VS Code screen, you will now see a RAW Query Properties pane. In here there is the metadata for the structure that’s just been returned in the latest query:

RAW Query Properties pane

Creating a Typealias

You can copy this structure which is very useful when you want to create a typealias.

For instance, I’ve just clicked ‘copy’ above, and pasted it below, defined a typealias t and then used it in the equivalent query below:

Query with a typealias

After this, it’s very easy to tailor the typealias and return just the fields we want, or rearranging the query to return a different structure.

Running a “Describe” on a data source

We can run a describe command on any data source – this returns metadata about the structure in a similar way.

Perform a “Ctrl+Click” (on PC) or “Cmd+Click” (on Mac) on a source (see below)

Doing that in this example brings up two sub-panes in the Data Preview, the left-hand sub-pane is a Data Description, showing format, encoding, confidence, and other metadata. The right-hand sub-pane is the RQL type of the source, which again can be copied to the clipboard as above

running a DESCRIBE on a data source

Querying large amounts of data

If you have larger amounts of data, for instance the 15 Million rows of NYC Parking Violations, our system is smart with the results. We perform pagination (see the pages at the bottom right corner) and bring the pages into memory as and when you need them (i.e. when you click):

Pagination in the Table view

In the Tree View, we split collections into 100 rows at a time. You can then open up any sub-section and it brings those into memory space on demand too:

Pagination in the Tree view

Lastly, in the Raw view there is a loading of data when you scroll down to 90% of the current screen, this means again that performance is better and memory is minimised in this view too.

Cancelling a long-running query

Finally, you can kill a long-running query under the spinning ‘Executing Query’ wheel:


Wrap Up

That’s it for this blog post – We hope you understand how to use some of these features better – they help to make RAW better to query and show large and complex datasets, and use the return types too. Any suggestions are welcome – please contact us if you have ideas, or share on the community

Jeremy Posner, VP Product & Solutions, RAW Labs.


Want to learn more?

Learn More

VIDEOS

Product Walkthrough

Walkthrough of Major Features, using a worked example to show both Data Consumer and Data Producer perspectives. Code accompanying this demo can be found on GitHub. You can also See Our Demo Catalog Here, where you can register for your...

Blog

Powerful new SEO & SEM Data Products with RAW

Create powerful SEO/SEM data products with RAW. This example compares historical search volumes for two keywords / brands over a time period for trend analysis.

Blog

Defining and Running Tests for API endpoints

We show how easy it is to create an API test suite using just YAML for your RAW APIs inside VS Code using our RAW Extension: Define tests, execute them and fix your code.