jq turns JSON from APIs, cloud CLIs, and deployment tools into data that is easy to inspect, filter, reshape, validate, and pass to the next shell command. This reference explains the core jq model, compares it with common alternatives, and provides reusable recipes for nested objects, arrays, grouping, joins, CSV output, and safer error handling.
Overview
JSON is convenient for machines but often awkward to read in a terminal. A response may contain several levels of objects and arrays, while the value you need is buried inside a long payload. jq is a command-line JSON processor that lets you describe the transformation as a filter.
A basic command reads JSON from standard input and writes the result to standard output:
curl -s https://api.example.test/services | jq '.items[] | .name'
The filter .items[] | .name walks into the items array, produces one item at a time, and selects each item's name. The result is still valid JSON unless you explicitly request raw text with -r.
That distinction matters in automation. Use normal jq output when another JSON-aware program will consume the result. Use jq -r when you need plain text for a shell variable, a loop, or a command argument. Always quote the filter in the shell so characters such as $, brackets, pipes, and parentheses are passed to jq rather than interpreted by the shell.
response=$(curl -sS https://api.example.test/config)
region=$(printf '%s' "$response" | jq -r '.region // "unknown"')
printf 'Region: %s\n' "$region"
This approach is useful in developer tools and DevOps workflows: inspecting deployment output, comparing configuration snapshots, extracting resource identifiers, or converting an API response into input for a later command.
How to compare options
jq is not the only way to parse JSON from a shell. The right choice depends on the shape of the task and the reliability requirements around it.
- Use jq for short, composable transformations. It is a strong fit when the input is JSON and the output can be expressed as a filter: select records, rename fields, calculate values, or emit a compact report.
- Use a language runtime for larger programs. Python, JavaScript, Go, or another general-purpose language is usually easier to maintain when the workflow needs multiple functions, network calls, tests, complex state, or detailed error messages.
- Use the provider CLI's query feature when it is sufficient. Many cloud CLIs can filter or format their own response. Keeping a simple projection inside the CLI can reduce piping, but jq is often more flexible when combining outputs from different commands or APIs.
- Use a YAML-aware tool for YAML. jq expects JSON. Tools designed for YAML can preserve or interpret YAML-specific syntax more appropriately. Convert formats deliberately rather than assuming that a JSON filter will safely handle every YAML document.
Compare options using five practical questions: Is the input guaranteed to be JSON? Will the filter be reused? Does the result need to remain machine-readable? How should missing fields and malformed input behave? Can another team member understand and test the command later?
For a one-off inspection, a concise jq filter is often clearer than writing a temporary script. For a production pipeline, treat the filter as code: document the expected input shape, test representative fixtures, and make failure behavior explicit.
Feature-by-feature breakdown
Objects, fields, and defaults
Use dot notation to select object fields:
jq '.metadata.name'
jq '.metadata.labels.environment'
Bracket notation is useful when a key contains punctuation or when the key is dynamic:
jq '.metadata["app.kubernetes.io/name"]'
jq --arg key "$FIELD" '.[$key]'
The alternative operator, //, supplies a fallback for null or absent values:
jq '.owner // "unassigned"'
jq '.spec.replicas // 1'
Arrays and selection
Use [] to iterate through an array. A selection can then keep only matching records:
jq '.deployments[] | select(.status == "ready") | .name'
To return a filtered array rather than separate output values, wrap the expression in brackets:
jq '[.deployments[] | select(.status == "ready")]'
Common array operations include length, sort_by(.field), unique, map(...), and any or all for checks. For example:
jq '.servers | map({name, address}) | sort_by(.name)'
jq 'any(.checks[]; .result == "failed")'
Reshaping objects
Object construction is useful when an API response contains more information than the next command needs:
jq '.items[] | {id: .id, label: .displayName, active: (.state == "active")}'
When the input field names already match the desired output, shorthand keys keep the filter readable:
jq '.items[] | {id, name, region}'
String interpolation creates human-readable lines, while -r removes JSON string quotes:
jq -r '.items[] | "\(.name)\t\(.status)"'
Grouping, counting, and joins
To group records by a field, sort them by the same field first, then apply group_by:
jq 'sort_by(.team) | group_by(.team) | map({team: .[0].team, count: length})'
To combine two arrays by a shared identifier, build an index from one side and look it up from the other:
jq '
.users as $users
| .projects
| map(. + {owner: ($users | map({key: .id, value: .}) | from_entries)[.ownerId]})
'
For large or frequently reused joins, a named helper or a general-purpose script may be easier to review. Prefer clarity over compressing a complex transformation into one line.
CSV and shell-safe output
Convert selected fields into an array and use @csv:
jq -r '.items[] | [.id, .name, .status] | @csv'
Use @tsv for tab-separated output. These formatters handle quoting more safely than manually concatenating commas, but downstream consumers still need to know whether a header row is present. Add one explicitly when required:
jq -r '(["id", "name", "status"] | @csv), (.items[] | [.id, .name, .status] | @csv)'
Validation and errors
To check whether a file is valid JSON without printing it, use jq's exit status:
if jq empty payload.json; then
echo "valid JSON"
else
echo "invalid JSON" >&2
exit 1
fi
For required fields, make the condition explicit rather than allowing an empty result to pass unnoticed:
jq -e 'has("id") and (.id | type == "string")' payload.json >/dev/null
Use try ... catch when a recoverable conversion is expected, but avoid hiding every error. A pipeline that silently turns a malformed response into an empty value can produce a more dangerous result than a failed pipeline.
Best fit by scenario
Inspecting an API response: start with curl -sS, pipe to jq, and narrow the response incrementally. First print the top-level keys, then inspect the relevant object or array. This makes changed response shapes easier to diagnose.
curl -sS "$API_URL" | jq 'keys'
curl -sS "$API_URL" | jq '.data | keys'
Working with cloud or Kubernetes output: request JSON output from the CLI when that option is available, then use jq for fields that the built-in formatter does not expose. Keep provider-specific commands separate from generic transformations so each step is easier to troubleshoot. If the command is investigating a pod or deployment, pair the extracted values with a broader Kubernetes troubleshooting guide.
Building a CI/CD check: pin down the expected shape with a fixture and use jq -e for assertions. Check both success and failure cases. For scheduled jobs, keep the JSON transformation independent from the schedule definition; the cron expression guide can help review the scheduling layer separately.
Debugging an HTTP failure: preserve the status code, headers, and body instead of piping only the body into jq. A valid JSON error document does not mean the request succeeded. Use the HTTP API error troubleshooting reference when the response needs to be diagnosed as a whole.
Comparing configuration snapshots: normalize each document before comparing it. Select stable fields, sort arrays where order is not meaningful, and remove volatile timestamps or request identifiers. Store the normalized output as an artifact so another operator can reproduce the comparison.
When to revisit
Revisit a jq workflow whenever the upstream JSON schema changes, a cloud CLI changes its output mode, or a pipeline begins receiving null, missing, or differently typed fields. A filter that worked against .items[] may fail when an API starts returning a single object, an omitted field, or an error envelope.
Review filters during dependency upgrades as well. jq versions and shell environments can differ in supported features, formatting behavior, and quoting constraints. Check the version available on developer laptops, CI runners, containers, and production hosts rather than assuming they are identical.
A practical maintenance checklist is:
- Record a representative success response and at least one error response.
- Test missing fields, empty arrays, null values, and unexpected types.
- Confirm whether output is JSON, raw text, CSV, or TSV before changing a consumer.
- Run the filter with the same shell and jq version used by automation.
- Review any upstream API or CLI release notes that affect output structure.
- Prefer a clear failure over silently producing incomplete deployment or configuration data.
When a filter becomes too long to explain in a code review, move the surrounding workflow into a tested script and keep jq for focused transformations. jq is most valuable when its input contract, output contract, and failure behavior are visible to the next person maintaining the command.