How to debug http (also https) communication

Intro

If you ever wondered how to debug some requests that are being made to or from IRIS, well here is a little tutorial on how to do that.

During a complex project, usually you get the specifications and implement the communication between IRIS and other things based on that. But from the paper to the real world there's usually a huge gap and you need to know why you are receiving an error on a parameter, on a header, you are not receiving the data and so on.

If the connection is a plain http connection there's no problem, you can always fire up tcpdump and capure the traffic, but what about https communication?

What about having a clean web interface, something that you start and then the developer can look at that portal when they like?

If you ever been in this situation, a solution can be easily done with mitm proxy ( https://mitmproxy.org/ ).

This program support to be a proxy (you can configure it in the BO for example), a transparent proxy (what i like), upstream proxy (transparent proxy that forward the connection to another proxy),...

Setup

To set it up

• simply download the file from the download page
• copy it on the server with iris (same server is easier to setup, but it can be on another server too).
• Once copied extract the archive in a directory (i usually call it mitm)

Start MITMProxy

I'll suggest to have tmux or screen installed too and to make a script for the start so that it can be interrupted.

#!/bin/bash

sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv6.conf.all.forwarding=1
sysctl -w net.ipv4.conf.all.send_redirects=0


iptables -t nat -A OUTPUT -d THE_DESTINATION_IP -p tcp --dport 443 -j REDIRECT --to-port 46465

SSLKEYLOGFILE="$PWD/.mitmproxy/sslkeylogfile.txt" ./mitmweb --mode transparent --no-anticache --listen-port 46465 --web-host THE_SERVER_IP --web-port 46464 --ssl-insecure --server -w /tmp/`date +%Y%m%d_%H%M_mitmproxy.dump` 

iptables -t nat -D OUTPUT -d THE_DESTINATION_IP -p tcp --dport 443 -j REDIRECT --to-port 46465

firewall-cmd --reload

sysctl -w net.ipv4.ip_forward=0
sysctl -w net.ipv6.conf.all.forwarding=0
sysctl -w net.ipv4.conf.all.send_redirects=1

To start it

• open tmux or screen
• start the script

MITMProxy Running

Now you can access the web page (starting mitm will give you the link) and check all the requests trapped in the "sniffer".

You can select the single request, look at the headers and body, look at the reply and so on

Stop MITMProxy

• Just recover the tmux or screen session
• interrupt the script with ctrl+c

Finishing up

This is a really powerful tool, it can be used in many many ways and it's really useful helping in the communication debug.

The doc is really well written and it can be found on the site: https://docs.mitmproxy.org/stable/

It helped a lot in a lof of situation in the last 2 years so i've written this article hoping to help other with the same needs.

FAQ

This tool name is MITM that stands for "Man In The Middle" is this a bad thing?

No... and yes. MITM it's a common cybersecurity practice that can be used for good things (like the one explained here), but also for very bad things, so keep it safe, in a development environment. We are stopping at the surface of what this tool can do, it can rewrite part or totally the request/response and a lot of other dirty things.

SSL connection "cannot" be sniffed but we are viewing them in this tool, how is this possible?

The MITMProxy does a MITM "attack" (whaaaat?) so:

• your ssl connection starts from your IRIS
• goes to MITM Proxy that says "hey, i'm the destination server!"
• SSL connection is terminated there and the request is decrypted and showed in MITMProxy
• the proxy contact the destination server saying "hey, i'm the source server"
• the proxy sends the request in https

Yes, if everything is setup with the correct ssl certificates, the results are awesome,