How to use Lua scripts with qsv

Created by datHere Support, Modified on Thu, 02 May 2024 at 08:00 AM by datHere Support


Introduction:

qsv offers a wide range of commands for the endless wrangling applications. But what if you encounter a specific operation that requires a tailored solution? —qsv has you covered with its own domain-specific language for data wrangling: Lua.


Imagine you're using the lua command for a simple operation like this:

qsv luau map c "a + b" (where a, b, and c = column names).

However, as your data wrangling needs grow more complex, typing out long scripts on the command line becomes tedious.


This is where Lua scripts come into play. By using the "file:" prefix or the ".lua/.luau" file extension, you can read non-trivial scripts directly from the filesystem. These scripts, often referred to as 'Recipes,' offer a broader application and streamline the process of handling intricate data operations.

Your qsv command now looks more like this

qsv luau map 'Age' age.luau filename.csv -o newfile.csv

This guide introduces you to the fundamentals of Lua scripts with qsv, providing practical examples for better understanding for the first time user. Lua scripts offer a powerful way to extend the capabilities of qsv, enabling you to automate repetitive tasks, perform complex data transformations, and unlock new insights from your csv files

Video Guide:


Instructions:

Using Lua scripts with qsv can be summarized in 3 simple steps.

  • Preparation
  • Navigation
  • Execution

Step 1: Prepare

  • Ensure familiarity with the dataset to be utilized and define the desired outcome. In this context, the objective is to compute the running total for the "Amount" field.

  • Subsequently, a Lua recipe is crafted with the purpose of facilitating the computation of the running total. This tailored script is designed to streamline the process and ensure accurate calculations.


Step 2: Navigation

  • Open up your terminal and navigate to the directory where your dataset is stored.
    cd <file-path>  

Step 3: Execution

  • Next , using the qsv command we'll type in the command, qsv, new column name, in this case "Running total", the filename of the Lua recipe, file:test2.lua and the csv, data.csv
    qsv luau map "Running total" file:test2.lua data.csv 
  • qsv luau map "Running total" file:test2.lua data.csv | qsv table

    We can utilize the other qsv commands like qsv table here so we can get a real time preview of our data

  • qsv luau map "Running total" file:<Path of this recipe> test2.lua data.csv -o newfile.csv

    Here a new file is created by qsv which contains the changes made by the recipe


If the script and the dataset are saved in the same directory, the 'file:' keyword isn't necessary


By following the outlined steps, from gathering data to crafting Lua scripts and utilizing qsv commands, users can efficiently manipulate data and achieve precise outcomes.



Additional Links 

Learn more about Lua command here

More Articles on qsv Recipes

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select atleast one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article