3
Report Server
wanghongenpin edited this page 2025-10-26 21:59:17 +08:00

Reporting Service — Usage Guide

Overview

This document describes how to receive and handle single-request reports from ProxyPin. Reports are typically POSTed as JSON to your HTTP service. The reporting format uses a single HAR Entry: each report contains exactly one HAR entry object with a structure similar to:

{
  "startedDateTime": "2025-10-25T12:34:56.789Z",
  "time": 123,
  "request": { /* request part */ },
  "response": { /* response part */ },
  "timings": { /* timing info */ }
}

The server only needs to accept that entry (you may save the entry directly or wrap it into a standard HAR log object before saving). In the example below the server wraps the received entry into a HAR log containing a single entry and saves it as a JSON file, which can be opened or imported by HAR tools (DevTools / Charles / mitmproxy).

Note: Clients will use Content-Type: application/json; if the client enables compression (gzip), the request will include Content-Encoding: gzip. The server must decompress before parsing JSON. In addition to sending request and response data to the server, ProxyPin also includes the following data in the request header:

X-Report-Name: The name of the matching rule.

Request & Response Fields (detailed examples)

Below are more complete request and response field examples (as found inside a single entry) and brief explanations for each field. Use these structures as a reference for backend parsing and storage.

Request example

"request": {
  "method": "POST",
  "url": "https://api.example.com/v1/upload?user=1",
  "httpVersion": "HTTP/1.1",
  "cookies": [],
  "headers": [
    { "name": "Host", "value": "api.example.com" },
    { "name": "Content-Type", "value": "application/json; charset=utf-8" },
    { "name": "Authorization", "value": "Bearer abcdef" }
  ],
  "queryString": [
    { "name": "user", "value": "1" }
  ],
  "headersSize": -1,
  "bodySize": 1234,
  "postData": {
    "mimeType": "application/json",
    "text": "{\"name\":\"Alice\",\"age\":30}",
    "params": []
  }
}

Field notes (key points)

  • method: HTTP method (GET/POST/...).
  • url: Full request URL (scheme, host, path, query string).
  • httpVersion: Protocol version, e.g. "HTTP/1.1" or "HTTP/2.0".
  • headers: Array of header objects (HAR standard). You may map this to a dictionary or keep the array to preserve duplicate headers.
  • queryString: Array of query parameters (name/value).
  • headersSize: Size of the request headers in bytes; set -1 if unknown.
  • bodySize: Size of the original body in bytes; set -1 if unknown.
  • postData: If present, contains mimeType and text. Binary bodies are usually provided as base64 or as a preview, not raw bytes.
    • mimeType: e.g. application/json, multipart/form-data.
    • text: Body as text (may be JSON string). If binary, it might be a base64 string and accompanied with encoding: 'base64'.
    • params: For form data, an array of fields (optional).

The postData.text in the example is a JSON string that can be parsed directly. For large files, prefer client-side chunking or uploading only a preview/meta.

Response example

"response": {
  "status": 200,
  "statusText": "OK",
  "httpVersion": "HTTP/1.1",
  "cookies": [],
  "headers": [
    { "name": "Content-Type", "value": "application/json; charset=utf-8" },
    { "name": "Content-Encoding", "value": "gzip" }
  ],
  "content": {
    "size": 2048,
    "mimeType": "application/json",
    "text": "{\"result\":\"ok\",\"id\":123}"
  },
  "redirectURL": "",
  "headersSize": -1,
  "bodySize": 2048
}

Response notes (key points)

  • status / statusText: HTTP status code and text.
  • headers: Response headers as name/value pairs. Note Content-Encoding (gzip/br/zstd) — the reported content is typically a decoded preview, so content.text is decoded text or base64.
  • content.size: Original size of the response body in bytes.
  • content.mimeType: MIME type string.
  • content.text: Text preview or full text of the response body. If binary, this may be a base64 string with encoding: 'base64'.
  • bodySize: Similar to content.size, or -1 if unknown.

Clients often truncate large bodies (e.g., upload only the first 256KB preview) or upload binary previews as base64. The server should consider:

  • whether to decode base64 and restore a binary file (may consume significant disk space);
  • whether to enforce a maximum saved size;
  • using asynchronous storage/queueing to avoid blocking request handlers in production.

Python (Flask) server example

This minimal example decompresses (if needed), parses JSON, validates that the payload is a single HAR entry, wraps it into a standard HAR log and saves it to received_har/.

Setup

python3 -m venv venv
source venv/bin/activate
pip install flask

Save as report_server.py

# report_server.py
import os
import gzip
import json
from datetime import datetime
from flask import Flask, request, jsonify

app = Flask(__name__)
OUT_DIR = 'received_har'
os.makedirs(OUT_DIR, exist_ok=True)


def try_decompress(body: bytes, encoding: str):
    if not encoding:
        return body
    enc = encoding.lower()
    if 'gzip' in enc:
        try:
            return gzip.decompress(body)
        except Exception as e:
            raise RuntimeError(f'gzip decompress error: {e}')
    return body


@app.route('/report', methods=['POST'])
def report():
    try:
        raw = request.get_data()
        content_encoding = request.headers.get('Content-Encoding', '') or ''

        # decompress if needed
        try:
            body_bytes = try_decompress(raw, content_encoding)
        except Exception as e:
            return jsonify({'ok': False, 'error': f'decompress failed: {e}'}), 400

        # parse JSON
        try:
            payload = json.loads(body_bytes.decode('utf-8', errors='replace'))
        except Exception as e:
            return jsonify({'ok': False, 'error': f'json decode failed: {e}'}), 400

        # validate single HAR entry (must include request or response)
        if not isinstance(payload, dict) or ('request' not in payload and 'response' not in payload):
            return jsonify({'ok': False, 'error': 'expected a single HAR entry with request or response field'}), 400

        # wrap into a HAR log for compatibility
        har = {
            'log': {
                'version': '1.2',
                'creator': {'name': 'ProxyPin-report-server', 'version': '1.0'},
                'entries': [payload]
            }
        }

        # persist to file
        ts = datetime.utcnow().strftime('%Y%m%dT%H%M%S%f')[:-3]
        filename = os.path.join(OUT_DIR, f'har_{ts}.har')
        with open(filename, 'w', encoding='utf-8') as f:
            json.dump(har, f, ensure_ascii=False, indent=2)

        return jsonify({'ok': True, 'saved': filename}), 200

    except Exception as e:
        return jsonify({'ok': False, 'error': str(e)}), 500


if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Run

python report_server.py

curl examples (single entry)

Assume entry.json contains the single HAR entry shown above.

Uncompressed upload:

curl -X POST http://localhost:8080/report \
  -H "Content-Type: application/json" \
  --data-binary @entry.json

GZIP compressed upload:

gzip -c entry.json > entry.json.gz

curl -X POST http://localhost:8080/report \
  -H "Content-Type: application/json" \
  -H "Content-Encoding: gzip" \
  --data-binary @entry.json.gz

Quick inline test (minimal entry):

echo '{"startedDateTime":"2025-10-25T12:34:56.789Z","time":1,"request":{},"response":{},"timings":{}}' \
  | curl -X POST http://localhost:8080/report -H "Content-Type: application/json" -d @-

This is a translation of the user's Chinese doc. The user requested the English file be provided directly. The user also provided a desktop UI file but didn't ask for edits to it.

Please let me know if you want the English file saved into the repository; I can create it at docs/reporting_service_en.md.