Key-value Logs

Key-value is a simple structured data format that stores a collection of key and value pairs. Some applications use key-value formatted logs for logging, and you can fetch values from these logs by specifying the respective key in a log pattern. This pattern improves the search-filter capabilities and parses out the values seamlessly.

Site24x7 helps parse your key-value logs. Additionally, Site24x7 supports the combination of a normal log with key-value log patterns. Get more information from these logs using Site24x7 AppLogs.

Getting started

Log in to your Site24x7 account.
Download and install the Site24x7 Server monitoring agent (Windows | Linux).
Go to Admin > AppLogs > Log Types > Add Log Type.

You can refer to this Adding a Log Type help document for more information.

Key-value log pattern

Sample log

date="2014-06-18 11:57:46,719" at=info method=POST path="/en/admin/post/1/edit?k0=v0&k1=v1" host=helloworld-symfony.herokuapp.com request_id=e8843b25-3587-4229-a430-c93360a0e89f fwd="121.24.53.11" dyno=web.1 connect=1ms service=243ms status=302 bytes=559 protocol=https

Log pattern

The log pattern should start with the keyword keyvalue and contain a minimum of one date field. The following is the sample pattern defined by Site24x7 to parse the above key-value logs:

keyvalue $date:date:yyyy-MM-dd HH:mm:ss,SSS$ $at$ $method$ $path$ $host$ $request_id as requestid$ $fwd$ $dyno$ $connect$ $service$ $status:number$ $bytes:number$ $protocol$

If you wish to change the field name, say from request_id to requestid, use the as keyword. Also, if you want to exclude a field, you can remove it from the log pattern.

This log is separated into fields, each of which will take its respective value, and is then uploaded to Site24x7.

Field name	Field value
date	2014-06-18 11:57:46,719
at	info
method	POST
path	/en/admin/post/1/edit?k0=v0&k1=v1
host	helloworld-symfony.herokuapp.com
requestid	e8843b25-3587-4229-a430-c93360a0e89f
fwd	121.24.53.11
dyno	web.1
connect	1ms
service	243ms
status	302
bytes	559
protocol	https

Combination of normal logs with key-value pattern logs

Sample log

2012-02-07T09:43:06.123456+00:00 heroku[router]: at=info method=GET path="/stylesheets/dev-center/library.css" host=devcenter.heroku.com fwd="204.204.204.204" dyno=web.5 connect=1ms service=18ms status=200 bytes=13

Log pattern

The following log pattern contains a combination of normal and key-value logs:

$DateTime:date$ heroku[router]: $Message:keyvalue:$at$$method$ $path$ $host$ $fwd$ $dyno$ $connect$ $service$ $status:number$ $bytes:number$$

DateTime and heroku are the field names in a normal log pattern, whereas the field names followed by the Message:keyvalue keyword follow the key-value log pattern.

Field name	Field value
DateTime	2012-02-07T09:43:06.123456+00:00
at	info
method	GET
path	/stylesheets/dev-center/library.css
host	devcenter.heroku.com
fwd	204.204.204.204
dyno	web.5
connect	1ms
service	18ms
status	200
bytes	13

Related log types

JSON logs
XML logs