jq is a powerful command line utility that can be used for various types of JSON data processing. This article will briefly introduce some basic jq command operations, and include a few hands-on practice exercise.
For practice, download this sample JSON file from github or use wget command to download the file to your machine.
wget https://raw.githubusercontent.com/arunasoftware/public-blog-assets/master/jq-example-1.json ./
You will notice that the entire file contains a single line of long json which is quite hard to read.
See pretty JSON output
jq '.' jq-example-1.json
By default jq will display a pretty format JSON that’s much easier to read. Everything inside the single quote is called filter. In this example, the filter “.” is instructing jq to output everything from the input.
See Compact JSON
It is still possible to view the output in a compact format. For that, use the option -c
jq -c '.' jq-example-1.json
Select value of a particular key
jq '.university.name' jq-example-1.json
This will select just the value of the name of the university, which is "JQ University"
Get the total count of the students
jq '.university.students.totalCount' jq-example-1.json
It will display 100
How many departments are there?
jq '.university.departments | length' jq-example-1.json
Here, the output of departments is piped to to length filter which returns the length of the array, which is 2.
How many keys are present altogether?
jq '.university | length' jq-example-1.json
The length filter does not always require an array. It can also display the number of keys present under the given key, which is 5.
What are the department names?
jq '.university.departments[].name' jq-example-1.json
This will display just just Computer Science and Physics
Display the output just for Physics department
jq '.university.departments[] | select(.name == "Physics")' jq-example-1.json
This will display just the block of Physics department.
{ "name": "Physics", "totalFaculty": 10, "head": "Mr Y.", "studentsCount": 40 }
As you can see, the filters in jq provide a programming abstraction that can be used for a wide range of data processing. You can read the manual page or help section of the command to dig into more details.
man jq
jq --help
Exercise Problems
- Pretty print the entire input JSON, but with the keys alphabetically sorted.
- For each department, output just the name and the head keys for it.
- For each department, output just the name and the head keys for it, but change head to to say chairman instead.
- Display just the second Department.