command line

jq – a powerful command line utility for processing JSON

by

jq is a powerful command line utility that can be used for various types of JSON data processing. This article will briefly introduce some basic jq command operations, and include a few hands-on practice exercise.

For practice, download this sample JSON file from github or use wget command to download the file to your machine.

wget https://raw.githubusercontent.com/arunasoftware/public-blog-assets/master/jq-example-1.json ./

You will notice that the entire file contains a single line of long json which is quite hard to read.

See pretty JSON output

jq '.' jq-example-1.json

By default jq will display a pretty format JSON that’s much easier to read. Everything inside the single quote is called filter. In this example, the filter “.” is instructing jq to output everything from the input.

See Compact JSON

It is still possible to view the output in a compact format. For that, use the option -c

jq -c '.' jq-example-1.json

Select value of a particular key

jq '.university.name' jq-example-1.json

This will select just the value of the name of the university, which is "JQ University"

Get the total count of the students

jq '.university.students.totalCount' jq-example-1.json

It will display 100

How many departments are there?

jq '.university.departments | length' jq-example-1.json

Here, the output of departments is piped to to length filter which returns the length of the array, which is 2.

How many keys are present altogether?

jq '.university | length' jq-example-1.json

The length filter does not always require an array. It can also display the number of keys present under the given key, which is 5.

What are the department names?

jq '.university.departments[].name' jq-example-1.json

This will display just just Computer Science and Physics

Display the output just for Physics department

jq '.university.departments[] | select(.name == "Physics")' jq-example-1.json

This will display just the block of Physics department.

{
  "name": "Physics",
  "totalFaculty": 10,
  "head": "Mr Y.",
  "studentsCount": 40
}

As you can see, the filters in jq provide a programming abstraction that can be used for a wide range of data processing. You can read the manual page or help section of the command to dig into more details.

man jq
jq --help

Exercise Problems

  1. Pretty print the entire input JSON, but with the keys alphabetically sorted.
  2. For each department, output just the name and the head keys for it.
  3. For each department, output just the name and the head keys for it, but change head to to say chairman instead.
  4. Display just the second Department.

1. jq . jq-example-1.json  --sort-keys
2. jq -c '.university.departments[] | {head,name}' jq-example-1.json
3. jq -c '.university.departments[] | {name,chairman:.head}' jq-example-1.json
4. jq -c '.university.departments[1]' jq-example-1.json