Transactions

Show end to end transaction visibility from the data AppFirst collects. You can start a transaction trace from a web servers url or an sql statement from a database via a transaction log source. You can also start a transaction trace from any process with socket data running on any server.

Available APIs

/api/transactions/

View, turn on/off, enable/disable transactions. If transactions are turned on, the appropriate logs sources are created to capture url and sql statements for transaction tracing. If transactions are disabled and transactions are on, log history is maintained, otherwise if transactions are off, logs will be deleted and history is lost.

GET

Examples

curl --user {EMAIL}:{API_KEY} https://wwws.appfirst.com/api/transactions
{
    "transactions_enabled": true, 
    "transactions_on": true
}

PUT

Set the transactions_on and transactions_enabled flag to turn transactions on or off, enabled or disabled.

Arguments

  • transactions_enabled: (true or false)
  • transactions_on: (true or false)

Arguments Example

curl --user {EMAIL}:{API_KEY} https://wwws.appfirst.com/api/transactions/ -X PUT -d "transactions_on=true&transactions_enabled=false"
{
    "transactions_enabled": false, 
    "transactions_on": true
}

/api/transactions/logs/

Attributes

id Unique ID for UserProfile in our system.
server_id The ID of the server this log is on.
source_type The type of the log, one of: AFSQL, AFURL.
source The full path to the log file.
limit Only collect up to this many messages from this log per minute.
url The URL to get more information about this item.

GET

Lists only logs related to transactions.

Arguments

  • limit (optional, default:2500, max:2500) – Sets the page size to a limit set by the user.
  • page (optional, default:0) – Retrieve the specific page of data of size limit.

Arguments Examples

  • limit=10 – will get the first page of the first 10 items.
  • limit=25&page=3 – will get the 4th page of size 25 items.
curl --user {EMAIL}:{API_KEY} https://wwws.appfirst.com/api/logs/

{
    "pagination": {
        "count": 34, 
        "next": null, 
        "previous": null
    }, 
    "data": [
        {
            "id": 392, 
            "server_id": 7, 
            "source_type": "AFSQL", 
            "source": "/var/log/af_sql.log", 
            "limit": 2000, 
            "url": "http://local.appfirst.com:8000/api/logs/392/"
        }, 
        {
            "id": 393, 
            "server_id": 7, 
            "source_type": "AFURL", 
            "source": "/var/log/af_url.log", 
            "limit": 2000, 
            "url": "http://local.appfirst.com:8000/api/logs/393/"
        },
}

/api/transactions/logs/{log_id}/detail/

Retrieves log messages for the given log. Each piece of log detail data has the following attributes:

data The log message.

Depending on the logs source_type (AFSQL, AFURL), the log messages will be formatted slightly different:

AFSQL


{
            "time": 1424215480, 
            "data": "[\"Data\", \"1\", \"PostgreSQL\", 1424215442.186389, 26688, 9801, 9, \"10.7.7.97:5432\", \"10.7.7.15:51943\", \"BEGIN; SET TRANSACTION ISOLATION LEVEL READ COMMITTED\"]", 
            "severity": "info", 
            "name": "/var/log/af_sql.log"
},

The data element is a stringified json object that consist of:
event, version, application, time, process id, parent process id, socket, my ip:my port, peer ip:peer port, sql statement

AFSQL


{
            "time": 1424215450, 
            "data": "[\"\", \"0\", \"Apache\", 1424215442.215066, 3303, 24610, -1, \"10.7.7.15:443\", \"10.7.7.185:52506\", \"GET /v2/api/server/polled_data/?pk=8 HTTP/1.1\"]", 
            "severity": "info", 
            "name": "/var/log/af_url.log"
},

The data element is a stringified json object that consist of:
event, version, application, time, process id, parent process id, socket, my ip:my port, peer ip:peer port, url

GET

Gets messages for the given log. It gets messages for up to “num” points starting from “end” and going back “start.” Note there can be multiple messages per minute. It also takes filter and severity parameters to restrict log messages that are returned.

Arguments

  • num (optional, default:1) – Retrieve up to this many number of points. Note that there can be gaps in the data if the server this log is on has an outage.
  • end (optional, default:most recent point) – Retrieve data from this timestamp backwards. If not given, it gets the most recent data.
  • start (optional) – Don’t retrieve any points before this date, if given. From V3, If both start and num are given, end will be assigned to start + time_step_in_epoch (default to be 60) * num.
  • time_step (optional, default:Minute) – Time step for the points, can only be ‘Minute’
  • filter (optional) – Only return messages which match this basic regular expression. If none given, it matches all messages.

Arguments Examples

  • num=3 – will get the messages from three most recent minutes of data.
  • num=1440&start=<12:00am today> – will get all of today’s data (which will probably be less than 1440 points).
  • num=3&end=1288584000 – will get the three minutes of data just before Nov. 11, 2010 UTC (which is 1288584000). If all the data exists, this will be 1288584000, 1288583940, and 1288583880. If the middle minute of data didn’t exist it would return 1288584000 and 1288583880.
  • num=30&end=1288584000&start=1288583940 – will get 2 minutes of data (if they exist) from 1288584000 and 1288583940.
  • num=3&time_step=Minute – will get the three most recent minutes of data.
  • filter=.*test[0-9] – will get messages from the last minute that match the expression (matchs: ‘test0′, ‘blah test9′; does NOT match: ‘testa’, ‘test9 blah’).
curl --user {EMAIL}:{API_KEY} https://wwws.appfirst.com/api/transactions/logs/399/detail/

{
    "pagination": {
        "count": 42, 
        "previous": null, 
        "next": null
    }, 
    "data": [
        {
            "time": 1424196190, 
            "data": "[\"\", \"0\", \"Apache\", 1424196130.426703, 3242, 24610, -1, \"10.7.7.15:443\", \"10.7.7.44:36538\", \"POST /SaveData.py/save_data HTTP/1.1\"]", 
            "severity": "info", 
            "name": "/var/log/af_url.log"
        }, 
        {
            "time": 1424196190, 
            "data": "[\"\", \"0\", \"Apache\", 1424196136.781867, 3303, 24610, -1, \"10.7.7.15:443\", \"10.7.7.99:58332\", \"POST /SaveData.py/save_data HTTP/1.1\"]", 
            "severity": "info", 
            "name": "/var/log/af_url.log"
        },
}

/api/transactions/log/

Returns detailed log data for a specific process between a start time and duration.

GET

Arguments

  • type: “sql” or “url”
  • lid: Log Id.
  • uid: Unique Process Id
  • start: The start time of the event
  • duration: The duration of the event

Arguments Example

curl --user {EMAIL}:{API_KEY} https://wwws.appfirst.com/api/transactions/log/?type=url&lid=399&uid=8_3279_13068630065000000&start=1424203529.313863&duration=0.02800893783569336

[
    {
        "type": "GET", 
        "time": 1424203529.766155, 
        "url": "/v2/api/server/?id=8 HTTP/1.1", 
        "ip": "10.7.7.15", 
        "ppid": 24610, 
        "pid": 3279, 
        "id": 49, 
        "port": 443
    }
]

/api/transactions/trace/

Returns a transaction trace of processes across one or multiple servers. Traces can start from a web servers url or an sql statement from a database. You can also start a transaction trace from any process with socket data running on any server.

Attributes

request

initialUid Unique process id of the process where the trace begins.
processes Number of processes found in the trace.
transactions Number of transactions found in the trace.

processes

id

Unique Id for the data set returned

args

The command line args used when starting this process.

avgResponseTime

The average response time of socket transactions in microseconds.

cpu

The CPU value in percent.

fileNum

The number of files accessed.

fileRead

Data read from files in bytes

fileWrite

Data written to files in bytes.

logId

The log id use in /api/transactions/log for detailed log data

logType

url, sql, or blank.

memory

The memory usage in bytes.

name

The name for this process.

netTransactNum

Average transaction count

pageFaults

The number of page faults.

pid

The operating system assigned process id (commonly called pid) for this process.

registryNum

The number of registries accessed.

responseNum

The number of socket responses sent (you can have multiple responses on a single network/socket connection).

serverHostName

The host name of the server this process is running on.

serverId

The ID of the server this process is running on.

serverIp

The Ip address of the server this process is running on.

serverNickName

The nickname of the server this process is running on.

socketNum

The number of network connections.

socketRead

Inbound network traffic in bytes.

socketWrite

Outbound network traffic in bytes.

threads

The number of threads.

time

The minute this data is for.

uid

Unique id for the process in our system which consists of server + “_” + pid + “_” + creation_time.

transactions

id

Unique Id for the data set returned.

clientDuration

Client side duration of the transaction.

clientIp

Client side Ip address of the server this process is running on.

clientPid

The operating system assigned process id (commonly called pid) for this client side process.

clientPort

Client side port number.

clientProcessId

Client side Id of the process from the process data object.

clientServerId

Client side Id of the server this process is running on.

clientStartTime

Client side start time of the transaction.

clientStatus

Client side socket status (Open, Closed)

clientUid

Unique id for the client side process in our system which consists of server + “_” + pid + “_” + creation_time.

serverDuration

Server side duration of the transaction.

serverIp

Server side Ip address of the server this process is running on.

serverPid

The operating system assigned process id (commonly called pid) for this server side process.

serverPort

Server side port number.

serverProcessId

Server side Id of the process from the process data object.

serverServerId

The ID of the server this server process is running on.

serverStartTime

Server side start time of the transaction.

serverStatus

Server side socket status (Open, Closed)

serverUid

Unique id for the server side process in our system which consists of server + “_” + pid + “_” + creation_time.

Arguments for a URL Start

  • sid: server_id from /api/transactions/logs.
  • pid: process id from /api/transactions/logs/{log_id}/detail data element.
  • time: time from /api/transactions/logs/{log_id}/detail data element.

Argument URL Example

curl --user {EMAIL}:{API_KEY} https://wwws.appfirst.com/api/transactions/trace/?time=1424198417.309437&sid=8&pid=3302

Arguments for a SQL Start

  • ip: peer ip from /api/transactions/logs/{log_id}/detail data element.
  • port: peer port from /api/transactions/logs/{log_id}/detail data element.
  • time: time from /api/transactions/logs/{log_id}/detail data element.

Argument SQL Example

curl --user {EMAIL}:{API_KEY} https://wwws.appfirst.com/api/transactions/trace/?time=1424198417.309437&ip=10.7.7.15&port=5432

Arguments for a Process Start

  • uid: uid from api/servers/[server id}/processes/.
  • time: integer time value (ex: 1234567890.123456).

Argument Process Example

curl --user {EMAIL}:{API_KEY} https://wwws.appfirst.com/api/transactions/trace/?time=1424198417.309437&uid=8_1931_13068691312000000
{
    "processes": [
        {
            "fileRead": 0, 
            "uid": "0_0_0", 
            "serverNickName": "External", 
            "pid": 0, 
            "logId": 0, 
            "logType": "none", 
            "registryNum": 0, 
            "id": 1, 
            "socketNum": 0, 
            "memory": 0, 
            "socketRead": 0, 
            "avgResponseTime": 0, 
            "fileWrite": 0, 
            "args": "", 
            "netTransactNum": 0, 
            "threads": 0, 
            "pageFaults": 0, 
            "socketWrite": 0, 
            "serverHostName": "External", 
            "name": "External", 
            "serverIp": "10.7.7.185", 
            "cpu": 0, 
            "flags": 0, 
            "time": 0, 
            "responseNum": 0, 
            "serverId": 0, 
            "fileNum": 0
        }, 
        {
            "fileRead": 6039, 
            "uid": "8_3302_13068630066000000", 
            "serverNickName": "dev0-frontend0", 
            "pid": 3302, 
            "logId": 399, 
            "logType": "url", 
            "registryNum": 0, 
            "id": 2, 
            "index": 0, 
            "socketNum": 15, 
            "memory": 90832896, 
            "socketRead": 414438, 
            "avgResponseTime": 93205.0, 
            "fileWrite": 1344, 
            "args": "/usr/sbin/httpd", 
            "netTransactNum": 14, 
            "threads": 2, 
            "pageFaults": 0, 
            "socketWrite": 253628, 
            "serverHostName": "dev0-frontend0", 
            "name": "httpd", 
            "serverIp": "10.7.7.15", 
            "fileNum": 15, 
            "flags": 0, 
            "time": 1424198400, 
            "responseNum": 14, 
            "serverId": 8, 
            "cpu": 0.15
        }, 
        {
            "fileRead": 4661100, 
            "uid": "9_9801_13066772869000000", 
            "serverNickName": "dev0-database0", 
            "pid": 9801, 
            "logId": 396, 
            "logType": "sql", 
            "registryNum": 0, 
            "id": 3, 
            "index": 0, 
            "socketNum": 116, 
            "memory": 34861056, 
            "socketRead": 2696919, 
            "avgResponseTime": 650.0, 
            "fileWrite": 1667198, 
            "args": "/usr/pgsql-9.0/bin/postmaster -p 5432 -D /var/lib/pgsql/9.0/data", 
            "netTransactNum": 2842, 
            "threads": 19, 
            "pageFaults": 0, 
            "socketWrite": 4314812, 
            "serverHostName": "dev0-database0", 
            "name": "postgres", 
            "serverIp": "10.7.7.97", 
            "fileNum": 5, 
            "flags": 0, 
            "time": 1424198400, 
            "responseNum": 2842, 
            "serverId": 9, 
            "cpu": 0.03
        }, 
        {
            "fileRead": 3408, 
            "uid": "13_27209_13068604865000000", 
            "serverNickName": "dev0-pipeline0", 
            "pid": 27209, 
            "logId": 401, 
            "logType": "url", 
            "registryNum": 0, 
            "id": 4, 
            "index": 0, 
            "socketNum": 9, 
            "memory": 37203968, 
            "socketRead": 329439, 
            "avgResponseTime": 159229.0, 
            "fileWrite": 503, 
            "args": "/usr/sbin/httpd", 
            "netTransactNum": 4, 
            "threads": 2, 
            "pageFaults": 0, 
            "socketWrite": 273691, 
            "serverHostName": "dev0-pipeline0", 
            "name": "httpd", 
            "serverIp": "10.7.7.99", 
            "fileNum": 14, 
            "flags": 0, 
            "time": 1424198400, 
            "responseNum": 4, 
            "serverId": 13, 
            "cpu": 0.43
        }, 
        {
            "fileRead": 31, 
            "uid": "11_14850_13066773074000000", 
            "serverNickName": "hbase1-hbregion1", 
            "pid": 14850, 
            "logId": 0, 
            "logType": null, 
            "registryNum": 0, 
            "id": 5, 
            "index": 0, 
            "socketNum": 62, 
            "memory": 442851328, 
            "socketRead": 5662175, 
            "avgResponseTime": 2719.0, 
            "fileWrite": 703, 
            "args": "/usr/java/default/bin/java -Dproc_thrift -XX:OnOutOfMemoryError=kill -9 %p -Xmx15000m -Dhdp.version=2.2.0.0-2041 -XX:+UseConcMarkSweepGC -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/hbase/hbase-hbase-thrift-hbase1-hbregion1.gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M -Dhbase.log.dir=/var/log/hbase -Dhbase.log.file=hbase-hbase-thrift-hbase1-hbregion1.log -Dhbase.home.dir=/usr/hdp/current/hbase-client/bin/.. -Dhbase.id.str=hbase -Dhbase.root.logger=INFO,RFA -Djava.library.path=:/usr/hdp/2.2.0.0-2041/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.2.0.0-2041/hadoop/lib/native -Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.thrift.ThriftServer -nonblocking start", 
            "netTransactNum": 506, 
            "threads": 78, 
            "pageFaults": 0, 
            "socketWrite": 5369271, 
            "serverHostName": "hbase1-hbregion1", 
            "name": "java", 
            "serverIp": "10.7.7.23", 
            "fileNum": 488, 
            "flags": 0, 
            "time": 1424198400, 
            "responseNum": 506, 
            "serverId": 11, 
            "cpu": 0.31
        }
    ], 
    “request”: {
        “initialUid”: “8_3302_13068630066000000″, 
        “processes”: 5, 
        “transactions”: 4
    }, 
    “transactions”: [
        {
            "serverProcessId": 2, 
            "clientPid": 0, 
            "clientPort": 63889, 
            "clientStartTime": 1424198416.832876, 
            "clientProcessId": 1, 
            "clientUid": "0_0_0", 
            "serverServerId": 8, 
            "serverDuration": 0.717584, 
            "clientServerId": 0, 
            "serverIp": "IPv6_Addr", 
            "serverStartTime": 1424198416.832876, 
            "serverStatus": "Closed", 
            "serverPid": 3302, 
            "serverPort": 443, 
            "clientDuration": 0.717584, 
            "serverUid": "8_3302_13068630066000000", 
            "clientIp": "10.7.7.185", 
            "id": 1, 
            "clientStatus": "Closed"
        }, 
        {
            "serverProcessId": 3, 
            "clientPid": 3302, 
            "clientPort": 51539, 
            "clientStartTime": 1424198416.840775, 
            "clientProcessId": 2, 
            "latency": 0.006294999999999995, 
            "clientUid": "8_3302_13068630066000000", 
            "serverServerId": 9, 
            "serverDuration": 0.462114, 
            "clientServerId": 8, 
            "serverIp": "10.7.7.97", 
            "serverStartTime": 1424198416.844679, 
            "serverStatus": "Closed", 
            "serverPid": 9801, 
            "serverPort": 5432, 
            "clientDuration": 0.468409, 
            "serverUid": "9_9801_13066772869000000", 
            "clientIp": "10.7.7.15", 
            "id": 2, 
            "clientStatus": "Closed"
        }, 
        {
            "serverProcessId": 4, 
            "clientPid": 3302, 
            "clientPort": 50066, 
            "clientStartTime": 1424198416.940447, 
            "clientProcessId": 2, 
            "latency": 0.004381999999999997, 
            "clientUid": "8_3302_13068630066000000", 
            "serverServerId": 13, 
            "serverDuration": 0.280402, 
            "clientServerId": 8, 
            "serverIp": "IPv6_Addr", 
            "serverStartTime": 1424198416.941469, 
            "serverStatus": "Closed", 
            "serverPid": 27209, 
            "serverPort": 80, 
            "clientDuration": 0.284784, 
            "serverUid": "13_27209_13068604865000000", 
            "clientIp": "10.7.7.15", 
            "id": 3, 
            "clientStatus": "Closed"
        }, 
        {
            "serverProcessId": 5, 
            "clientPid": 27209, 
            "clientPort": 41694, 
            "clientStartTime": 1424198416.944599, 
            "clientProcessId": 4, 
            "latency": -0.00019700000000000273, 
            "clientUid": "13_27209_13068604865000000", 
            "serverServerId": 11, 
            "serverDuration": 0.272738, 
            "clientServerId": 13, 
            "serverIp": "IPv6_Addr", 
            "serverStartTime": 1424198416.945602, 
            "serverStatus": "Closed", 
            "serverPid": 14850, 
            "serverPort": 9090, 
            "clientDuration": 0.272541, 
            "serverUid": "11_14850_13066773074000000", 
            "clientIp": "10.7.7.99", 
            "id": 4, 
            "clientStatus": "Closed"
        }
    ]
}

Example URL Trace

1. Find a server from “/api/servers” that is running a web application

{
            "id": 8, 
            "hostname": "dev0-frontend0", 
            "nickname": "dev0-frontend0", 
            "os": "Linux", 
            "created": 1421778935, 
            "running": false, 
            "capacity_cpu_freq": 3100, 
            "description": "", 
            "location_ip": "10.7.7.15", 
            "capacity_cpu_num": 2, 
            "capacity_mem": 4069523456, 
            "current_version": 103, 
            "os_detail": "RedHat 2.6.32-358.el6.x86_64", 
            "arch": "x86_64", 
            "guid": "dbe4912cd053e6c1a215c7827d0d2773", 
            "capacity_disks": {
                "/dev/xvda1": 484, 
                "/dev/mapper/VolGroup-lv_root": 25788
            }, 
            "network_interfaces": [
                {
                    "ip": [
                        "127.0.0.1"
                    ], 
                    "friendly_name": "lo", 
                    "name": "lo"
                }, 
                {
                    "ip": [
                        "10.7.7.15"
                    ], 
                    "friendly_name": "eth0", 
                    "name": "eth0"
                }
            ], 
            "url": "http://local.appfirst.com:8000/api/servers/8/"
        }

2. Get a list of log resources from “/api/transactions/logs” and find a log with server_id={id from servers} and source_type=”AFURL”

{
            "id": 399, 
            "server_id": 8, 
            "source_type": "AFURL", 
            "source": "/var/log/af_url.log", 
            "limit": 2000, 
            "url": "http://local.appfirst.com:8000/api/logs/399/"
},

3. Get a list of url’s from “/api/transactions/logs/399/detail/?end=1424274310” (399 is the id from the logs api). If you want to get the recent minute, you can exclude the end argument.

{
    "pagination": {
        "count": 32, 
        "previous": null, 
        "next": null
    }, 
    "data": [
        {
            "time": 1424274370, 
            "data": "[\"\", \"0\", \"Apache\", 1424274310.738288, 1631, 24610, -1, \"10.7.7.15:443\", \"10.7.7.99:35517\", \"POST /SaveData.py/save_data HTTP/1.1\"]", 
            "severity": "info", 
            "name": "/var/log/af_url.log"
        }, 
        {
            "time": 1424274370, 
            "data": "[\"\", \"0\", \"Apache\", 1424274366.050146, 1628, 24610, -1, \"10.7.7.15:443\", \"10.7.7.191:65234\", \"POST /SaveData.py/save_data HTTP/1.1\"]", 
            "severity": "info", 
            "name": "/var/log/af_url.log"
        }
    ]
}

4. Select one of the data elements from the log details and call “/api/transactions/trace/?time=1424274366.050146&sid=8&pid=1628”

Example SQL Trace

1. Find a server from “/api/servers” that is running a database application

{
            "id": 9, 
            "hostname": "dev0-database0", 
            "nickname": "dev0-database0", 
            "os": "Linux", 
            "created": 1421779167, 
            "running": false, 
            "capacity_cpu_freq": 3100, 
            "description": "", 
            "location_ip": "10.7.7.97", 
            "capacity_cpu_num": 2, 
            "capacity_mem": 3994025984, 
            "current_version": 103, 
            "os_detail": "RedHat 2.6.32-358.el6.x86_64", 
            "arch": "x86_64", 
            "guid": "82a043236341dbd1668b3e4cad0ac5f2", 
            "capacity_disks": {
                "/dev/xvda1": 484, 
                "/dev/mapper/VolGroup-lv_root": 25851
            }, 
            "network_interfaces": [
                {
                    "ip": [
                        "127.0.0.1"
                    ], 
                    "friendly_name": "lo", 
                    "name": "lo"
                }, 
                {
                    "ip": [
                        "10.7.7.97"
                    ], 
                    "friendly_name": "eth0", 
                    "name": "eth0"
                }
            ], 
            "url": "http://local.appfirst.com:8000/api/servers/9/"
        },

2. Get a list of log resources from “/api/transactions/logs” and find a log with server_id={id from servers} and source_type=”AFSQL”

{
            "id": 396, 
            "server_id": 9, 
            "source_type": "AFSQL", 
            "source": "/var/log/af_sql.log", 
            "limit": 2000, 
            "url": "http://local.appfirst.com:8000/api/logs/396/"
},

3. Get a list of sql statements from “/api/transactions/logs/396/detail/?end=1424274310” (396 is the id from the logs api). If you want to get the recent minute, you can exclude the end argument.

{
    "pagination": {
        "count": 32, 
        "previous": null, 
        "next": null
    }, 
    "data": [
        {
            "time": 1424275240, 
            "data": "[\"Data\", \"1\", \"PostgreSQL\", 1424275202.991453, 4240, 9801, 9, \"10.7.7.97:5432\", \"10.7.7.15:59758\", \"SELECT \\\"main_maintenancewindow\\\".\\\"id\\\", \\\"main_maintenancewindow\\\".\\\"tenant_id\\\", \\\"main_maintenancewindow\\\".\\\"start\\\", \\\"main_maintenancewindow\\\".\\\"end\\\", \\\"main_maintenancewindow\\\".\\\"started\\\", \\\"main_maintenancewindow\\\".\\\"ended\\\", \\\"main_maintenancewindow\\\".\\\"reason\\\", \\\"main_maintenancewindow\\\".\\\"deleted\\\", \\\"main_maintenancewindow\\\".\\\"server_count\\\" FROM \\\"main_maintenancewindow\\\" WHERE (\\\"main_maintenancewindow\\\".\\\"started\\\" = false  AND \\\"main_maintenancewindow\\\".\\\"start\\\" <= E'2015-02-18 16:00:02.989867' )\"]", 
            "severity": "info", 
            "name": "/var/log/af_sql.log"
        }, 
        {
            "time": 1424275240, 
            "data": "[\"Data\", \"1\", \"PostgreSQL\", 1424275202.992431, 4240, 9801, 9, \"10.7.7.97:5432\", \"10.7.7.15:59758\", \"SELECT \\\"main_maintenancewindow\\\".\\\"id\\\", \\\"main_maintenancewindow\\\".\\\"tenant_id\\\", \\\"main_maintenancewindow\\\".\\\"start\\\", \\\"main_maintenancewindow\\\".\\\"end\\\", \\\"main_maintenancewindow\\\".\\\"started\\\", \\\"main_maintenancewindow\\\".\\\"ended\\\", \\\"main_maintenancewindow\\\".\\\"reason\\\", \\\"main_maintenancewindow\\\".\\\"deleted\\\", \\\"main_maintenancewindow\\\".\\\"server_count\\\" FROM \\\"main_maintenancewindow\\\" WHERE (\\\"main_maintenancewindow\\\".\\\"ended\\\" = false  AND \\\"main_maintenancewindow\\\".\\\"end\\\" <= E'2015-02-18 16:00:02.989867' )\"]", 
            "severity": "info", 
            "name": "/var/log/af_sql.log"
        }, 
        {
            "time": 1424275240, 
            "data": "[\"Data\", \"1\", \"PostgreSQL\", 1424275203.304787, 4241, 9801, 9, \"10.7.7.97:5432\", \"10.7.7.15:59762\", \"SET DATESTYLE TO 'ISO'\"]", 
            "severity": "info", 
            "name": "/var/log/af_sql.log"
        },
    ]
}

4. Select one of the data elements from the log details and call “/api/transactions/trace/?time=1424275203.304787&ip=10.7.7.15&port=59762”

Example Process Trace

1. Get a server from “/api/servers/”

{
            "id": 13, 
            "hostname": "dev0-pipeline0", 
            "nickname": "dev0-pipeline0", 
            "os": "Linux", 
            "created": 1421779339, 
            "running": false, 
            "capacity_cpu_freq": 3100, 
            "description": "", 
            "location_ip": "10.7.7.99", 
            "capacity_cpu_num": 2, 
            "capacity_mem": 4018143232, 
            "current_version": 103, 
            "os_detail": "RedHat 2.6.32-358.el6.x86_64", 
            "arch": "x86_64", 
            "guid": "e94eabf55654e66a44fcfd68efe52145", 
            "capacity_disks": {
                "/dev/mapper/vg_dev0pipeline0-lv_root": 20795, 
                "/dev/sda1": 484
            }, 
            "network_interfaces": [
                {
                    "ip": [
                        "127.0.0.1"
                    ], 
                    "friendly_name": "lo", 
                    "name": "lo"
                }, 
                {
                    "ip": [
                        "10.7.7.99"
                    ], 
                    "friendly_name": "eth0", 
                    "name": "eth0"
                }
            ], 
            "url": "http://local.appfirst.com:8000/api/servers/13/"
        },

2. Get a list of processes for this server with “/api/servers/13/processes”

{
    "pagination": {
        "count": 47, 
        "previous": null, 
        "next": null
    }, 
    "data": [
        {
            "server_id": 13, 
            "uid": "13_1308_13066757683000000", 
            "args": "/sbin/mingetty /dev/tty4", 
            "pid": "1308", 
            "create_time": "13066757683000000", 
            "name": "mingetty"
        }, 
        {
            "server_id": 13, 
            "uid": "13_1256_13066757682000000", 
            "args": "ntpd -u ntp:ntp -p /var/run/ntpd.pid -g", 
            "pid": "1256", 
            "create_time": "13066757682000000", 
            "name": "ntpd"
        }, 
        {
            "server_id": 13, 
            "uid": "13_26940_13068691262000000", 
            "args": "/usr/sbin/httpd", 
            "pid": "26940", 
            "create_time": "13068691262000000", 
            "name": "httpd"
        }, 
        {
            "server_id": 13, 
            "uid": "13_3165_13066937827000000", 
            "args": "/usr/sbin/httpd", 
            "pid": "3165", 
            "create_time": "13066937827000000", 
            "name": "httpd"
        }, 
    ]
}

3. Select one of the data elements from the processes and call “/api/transactions/trace/?time=1424275203.000000&uid=13_3165_13066937827000000”