Wrangling some document conversion the other day, I ended up in a situation where I had the JSON I needed, but in a completely unreadable format. Luckily, this problem is very easily fixable .... when you know how. So today's post is a quick recap on how I did that using jq, a very handy command-line tool for working with JSON. For the impatient, here's the command:
cat posts.json | jq "." > better.json
In this post we'll look at the data I started with and what the different bits of the command to do help.
My data looked like this (actually I was working with an OpenAPI file, which was huge, so this is some other JSON data that will be easier to show the example with):
[{"title":"Open Standards for APIs","link":"https://beta.lornajane.net/posts/2024/open-standards-for-apis","date":"2024-10-09"},{"title":"Lint APIs with Redocly CLI","link":"https://beta.lornajane.net/posts/2024/lint-apis-with-redocly-cli","date":"2024-08-05"},{"title":"API Description Pipelines","link":"https://beta.lornajane.net/posts/2024/api-description-pipelines","date":"2024-06-18"},{"title":"Checking Links in Docs-As-Code Projects","link":"https://beta.lornajane.net/posts/2024/checking-links-in-docs-as-code-projects","date":"2024-04-27"}]
That's ... not very human friendly! I put this mess into a file called posts.json so I could more easily work with it.
Then I used the command that you saw above. Here is a quick recap of what each part does:
- cat is a command to print the contents of the file to stdout. It's a good way to get content into the start of a chain of commands.
- The | operator sends the output of the cat command into the next thing in the command.
- jq is a JSON tool, and the "." part is technically a filter - but it includes everything, so it's more like a not-filter. The output of jq is the same JSON that we put in, but it's pretty-printed.
- If we stopped at this point, we'd get pretty, human-readable JSON content to look at with colour highlights in the terminal. I use this mode a lot when I'm debugging APIs. But in this case, I wanted the cute output in a file.
- > sends the output of stdout to a destination that isn't the terminal, and in this case I made it a file better.json.
The data in the new file looks a lot better:
[ { "title": "Open Standards for APIs", "link": "https://beta.lornajane.net/posts/2024/open-standards-for-apis", "date": "2024-10-09" }, { "title": "Lint APIs with Redocly CLI", "link": "https://beta.lornajane.net/posts/2024/lint-apis-with-redocly-cli", "date": "2024-08-05" }, { "title": "API Description Pipelines", "link": "https://beta.lornajane.net/posts/2024/api-description-pipelines", "date": "2024-06-18" }, { "title": "Checking Links in Docs-As-Code Projects", "link": "https://beta.lornajane.net/posts/2024/checking-links-in-docs-as-code-projects", "date": "2024-04-27" } ]
Tools like jq are great to have to hand. What are the tools you can't manage without? I'd love to hear some suggestions!