Linux jq Command

Linux 命令大全Complete Linux Command Reference

jq is a lightweight and powerful command-line JSON processor. It allows you to parse, filter, transform, and format JSON data efficiently, and is especially suitable for processing JSON data in shell scripts.

Main Features of jq

  • Streaming processing: can handle JSON data of any size
  • Rich operators: provides a variety of operators to handle JSON data structures
  • Formatted output: can beautify JSON output for better readability
  • Cross-platform: runs on Linux, macOS, and Windows (via WSL)

Installing jq

On most Linux distributions, you can easily install jq via the package manager:

Examples

# Ubuntu/Debian
sudo apt-get install jq

# CentOS/RHEL
sudo yum install jq

# Fedora
sudo dnf install jq

# macOS (using Homebrew)
brew install jq

After installation, you can verify whether the installation was successful with the following command:

jq --version

Basic Syntax

The basic command format of jq is as follows:

jq [options]  [file...]

Common Options and Parameters

Option Description
-c Compact output (not beautified)
-r Output raw string (remove JSON quotes)
-s Read the entire input stream into an array
-M Disable color output
--arg Define a variable
--slurp Read multiple JSON objects into an array

Common Operation Examples

1. Basic JSON Parsing

Suppose we have a file nameddata.jsonwith the following content:

Example

{
  "name": "John Doe",
  "age": 30,
  "isActive": true,
  "address": {
    "street": "123 Main St",
    "city": "New York"
  },
  "hobbies": ["reading", "hiking", "coding"]
}

Get all content (beautified output)：

jq '.' data.json

Get a specific field：

Example

jq '.name' data.json
# Output: "John Doe"

Get a nested field：

Example

jq '.address.city' data.json
# Output: "New York"

2. Array Operations

Get array elements：

Example

jq '.hobbies[0]' data.json
# Output: "reading"

jq '.hobbies[1:3]' data.json
# Output: ["hiking", "coding"]

Array length：

Example

jq '.hobbies | length' data.json
# Output: 3

3. Conditional Filtering

Filter elements that meet the condition：

Example

# Suppose users.json contains an array of users
jq '.[] | select(.age > 25)' users.json

4. Data Transformation

Create a new object：

Example

jq '{fullName: .name, city: .address.city}' data.json
# Output: {"fullName": "John Doe", "city": "New York"}

Mathematical operations：

Example

jq '.age * 2' data.json
# Output: 60

5. Processing Multiple Files

jq -s '.[0] + .[1]' file1.json file2.json

Advanced Usage

1. Using Variables

jq --arg new_city "Boston" '.address.city = $new_city' data.json

2. Handling API Responses

curl -s https://api.example.com/users | jq '.[] | {name: .name, email: .email}'

3. Complex Data Transformation

Example

# Convert an array of objects to CSV format
jq -r '["Name", "Age"], (.[] | [.name, .age]) | @csv' users.json

4. Error Handling

Example

# Use try-catch to handle fields that may not exist
jq 'try .unknown_field catch "default value"' data.json

Real-world Application Scenarios

1. Log Analysis

Example

# Analyze log files in JSON format
cat app.log | jq 'select(.level == "error") | {time: .timestamp, message: .msg}'

2. Configuration Processing

Example

# Modify a value in the configuration file
jq '.config.timeout = 30' config.json > temp.json && mv temp.json config.json

3. Data Statistics

Example

# Calculate the average age of users
jq '[.[].age] | add / length' users.json

FAQ

1. How to handle large JSON files?

For very large JSON files, you can use streaming processing:

jq -n 'inputs | select(.value > 100)' hugefile.json

2. How to preserve special characters in JSON?

When using the-roption, special characters are escaped. If you need to output them as-is, you can:

jq -j '.' file.json

3. How to merge multiple JSON files?

jq -s 'add' file1.json file2.json

Summary

jq is a powerful tool for processing JSON data. Through this article, you should have mastered:

  1. Basic installation and usage of jq
  2. Common data query and filtering operations
  3. Advanced data transformation and processing techniques
  4. Usage methods in real-world application scenarios

To further improve your jq skills, you can:

  • Practice handling various complex JSON data structures
  • Try integrating jq into your shell scripts
  • Refer to the official jq manual to learn more advanced features

Remember, mastering jq can significantly improve your efficiency in processing JSON data on the command line!

Linux 命令大全Complete Linux Command Reference

Other Extensions