Parsing JSON fields in AWS Log Insights

This just took me too long to figure out to not write down somewhere!

Say you have a log of the form:

{
"timestamp":"2022-12-23T12:34:56Z",
"level":"error",
"message":"There was an error processing the request",
"request_id":"1234567890",
"user_id":"abcdefghij"
}

My reading of the log insights docs made me think this ought to work:

parse @message "'message': '*'" as logmessage
| sort @timestamp desc
| limit 20

Because you’re wanting to parse the message field out of the JSON. But it won’t, because rather than interpreting the JSON, Log Insights is doing a text match.

What you’d actually need is this:

parse @message '"message":"*"' as logmessage
| sort @timestamp desc
| limit 20

Note the orientation of the quote marks (we used double-quotes in the JSON so must use double-quotes in the match) and the spacing (we didn’t put a space between the key and the values in the log line).


Posted

in

by

Tags: