monocle v1.4.0: Remote BGP Search, PCH Archive Parsing, and RIB Correctness

Public BGP analysis is frequently constrained by both the source data and the machine doing the work. A single investigation may need to parse many MRT files, ingest Packet Clearing House’s Cisco-style route dumps, and compare an older route observation with the RIB and RPKI state from the same period. Doing all of that on an analyst’s working machine is impractical when it lacks the CPU, memory, disk, or persistent cache required for the data volume.

monocle v1.4.0 connects those requirements into one workflow. Its HTTP/SSE service moves downloads and parsing to a dedicated server, cloud VM, or container deployment while streaming matching results back to the working machine. The same release adds PCH text-dump parsing, reusable file-based filters, and RPKISPOOL-backed historical RPKI lookups, and corrects RIB and RIB-search behavior that could omit stable routes whose per-route timestamps predate the snapshot.

Install and run

The release can be installed with Cargo or Homebrew:

cargo install monocle --version 1.4.0
# or, on macOS and Linux with Homebrew
brew install monocle

monocle --version
# monocle 1.4.0

A multi-architecture bgpkit/monocle Docker image is available on Docker Hub. It provides a containerized deployment path for a dedicated search host or an orchestrated environment such as Kubernetes, where the service can retain its own database and MRT-file cache instead of relying on the workstation that submits a query.

docker run --rm \
  -p 8080:8080 \
  -v monocle-data:/data \
  bgpkit/monocle:1.4.0 \
  server --address 0.0.0.0 --port 8080

The persistent /data volume holds monocle’s SQLite-backed data. Add a separate cache mount and configure cache_dir when repeatedly searching historical MRT files. With the binary or container in place, the next decision is whether the working machine should execute the search itself or submit it to a dedicated service.

Run searches where the data and compute are

For a one-off query, local monocle search remains the shortest path. Longer investigations, repeated queries, and applications benefit from separating the working machine from the search host: the latter can keep the MRT cache and provide the CPU, memory, disk, and network capacity required to retrieve and parse public archives.

v1.4.0 replaces the previous WebSocket service with HTTP. Search results arrive as Server-Sent Events (SSE); health checks, metadata, lookups, and database management use ordinary REST endpoints. This lets the working machine submit a narrow query, while the remote service does the download and parsing work and streams matching elements and progress back as they become available.

# On the search host
monocle server --address 0.0.0.0 --port 8080

# A health check remains unauthenticated, even when API authentication is enabled.
curl http://127.0.0.1:8080/health
# OK

The service exposes GET /health, GET /api/v1/system/info, and POST /api/v1/search/stream, as well as endpoints for time, country, IP, RPKI, prefix-to-ASN, AS relationships, inspection, and database operations. Database-backed endpoints return 503 NOT_INITIALIZED until their data has been explicitly refreshed; a request does not trigger an implicit refresh.

The SSE endpoint accepts the same kinds of filters as the CLI. For example, this asks RIPE RIS collector rrc00 for updates concerning Cloudflare’s 1.1.1.0/24 in a ten-minute window:

curl -N \
  -H 'Accept: text/event-stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "filters": {
      "prefix": ["1.1.1.0/24"],
      "start_ts": "2025-06-01T00:00:00Z",
      "end_ts": "2025-06-01T00:10:00Z",
      "collector": "rrc00",
      "project": "riperis",
      "dump_type": "updates"
    },
    "batch_size": 100
  }' \
  http://127.0.0.1:8080/api/v1/search/stream

A stream begins with started, may emit progress and elements, and terminates with exactly one completed, cancelled, or error event. Its final event includes matched-element and source-file counts, compressed-byte metadata, rate, and the collectors and files that actually produced matches. Disconnecting cancels the work.

The server also makes its operating limits explicit. server_max_concurrent_searches defaults to 3 and excess requests receive HTTP 429. A bounded channel protects element batches from being dropped under backpressure; progress updates may be coalesced. server_max_search_batch_size, server_max_search_results, server_search_timeout_secs, and search_concurrency can be set in monocle.toml, through MONOCLE_* environment variables, or with server flags. Token authentication protects /api/v1/* while leaving /health available for container health checks.

The CLI can submit the same search to that remote service. This keeps scripts and output formatting local while moving downloads, parsing, and cached data to the server:

monocle search \
  -p 1.1.1.0/24 \
  -t 2025-06-01T00:00:00Z \
  -T 2025-06-01T00:10:00Z \
  -c rrc00 -P riperis \
  --remote-url https://monocle.example.net/api/v1/search/stream \
  --remote-token "$MONOCLE_REMOTE_TOKEN" \
  --format json-line

--remote-url consumes the SSE stream and uses monocle’s existing formatters. Invalid requests and pre-stream validation failures return HTTP errors; a stream that ends in error or cancelled, or drops without completed, makes the CLI exit non-zero. Remote execution addresses the compute constraint, but a useful search service must also accept the archive formats that researchers and operators actually have available.

Parse PCH’s Cisco-style BGP-table archives

MRT is the usual exchange format for public BGP data, but Packet Clearing House’s daily BGP routing archive publishes its route dumps as fixed-width Cisco-style text. The Cisco parser in v1.4.0 was added specifically so PCH data can enter the same monocle workflow without a separate conversion tool.

monocle parse detects the dump preamble and reads the prefix, AS path, next hop, metric, local preference, and origin. It also handles multiple paths for a prefix and layouts where an IPv6 prefix wraps onto its own line.

The following 2025-06-01 Accra collector snapshot is a 50 KB gzip-compressed example. monocle reads the remote archive directly:

monocle parse \
  https://downloads.pch.net/files/BGP_Routing_Data/BGP_ROUTES_V4/2025/06/route-collector.acc.pch.net/route-collector.acc.pch.net-ipv4_bgp_routes.2025.06.01.gz \
  --fields prefix,as_path,next_hop,local_pref,med \
  --format psv | head -5

The command produces rows such as:

1.179.112.0/20|15169 396982|196.201.2.118||0
2.16.77.0/24|35091 30997 20940|196.201.2.6||0
2.16.77.0/24|20940 20940|196.201.2.126||0
2.16.77.0/24|20940|196.201.2.160||0
2.16.77.0/24|37613 3741 37350 30997 20940 20940|196.201.2.32||0

The same input can be written as a TABLE_DUMP_V2 RIB for tools that expect MRT:

monocle parse router-show-ip-bgp.txt \
  --mrt-path router-snapshot.mrt \
  --mrt-type rib

Native support for the PCH text archives is the immediate goal of this release. Broader PCH archive support can follow as the archive formats and workflows are expanded. Once these archives can be queried alongside MRT data, the next practical problem is retaining the query definition as its scope grows.

Keep large filters out of shell arguments

An investigation that starts with one prefix often grows to hundreds or thousands. v1.4.0 adds --prefix-file and --filter-file to both parse and search, so that expanded query can be retained in a file instead of being reconstructed from a long shell command.

--prefix-file is newline-delimited text: blank lines and comments beginning with # are ignored. --filter-file is JSON and can hold prefixes, origin or peer ASNs, communities, an AS-path regular expression, element type, and time bounds.

{
  "prefixes": ["1.1.1.0/24", "2606:4700::/32"],
  "origin_asns": ["13335"],
  "peer_asns": ["174", "6939"],
  "include_sub": true,
  "start_ts": "2025-06-01T00:00:00Z",
  "end_ts": "2025-06-01T01:00:00Z"
}

monocle search \
  --filter-file cloudflare-investigation.json \
  -c rrc00 -P riperis \
  --format json-line

Values from a file merge with command-line values. Values within the same dimension are unioned; different dimensions remain ANDed. This supports versioned, reviewable query definitions without relying on oversized shell arguments. Those definitions can identify the messages around an event; answering what routing state existed at a particular instant requires reconstructing a RIB instead.

Reconstruct the RIB that existed at the requested time

A RIB dump is not a collection of routes learned only at the dump timestamp. Its per-route timestamps can be days or weeks earlier. Filtering a base RIB by the requested interval therefore discards stable routes and returns an incomplete state.

v1.4.0 fixes this in monocle rib and in monocle search --dump-type rib. The base RIB retains those stable entries, and a target timestamp exactly equal to a dump time selects that dump instead of an unnecessarily earlier snapshot. This avoids both incorrect output and needless update replay.

monocle rib 2025-06-01T00:00:00Z \
  -c rrc00 -P riperis \
  -p 1.1.1.0/24 \
  --format json-pretty

The command selects the latest RIB at or before the target timestamp, then replays updates through that exact time. The result answers a state question, “what routes were present then?”, rather than returning only messages whose timestamps fall in the selected interval. monocle rib also gains --use-cache, --cache-dir, and --fields in this release, which are useful for repeated or narrowed reconstruction runs. A reconstructed route still does not establish whether its origin was authorized at that time, so the final part of the workflow is a contemporaneous RPKI lookup.

Pair reconstructed routes with historical RPKI data

Route-origin analysis needs the authorization state from the same period. Applying current ROAs to an old routing event can produce the wrong conclusion. v1.4.0 makes RPKISPOOL the default historical source for monocle rpki roas and monocle rpki aspas; the format stores CCR snapshots in .tar.zst archives. The ripe and rpkiviews sources remain available. RPKIViews provides the historical RPKI data used by the rpkiviews source.

monocle rpki roas 1.1.1.0/24 \
  --date 2025-06-01 \
  --source rpkispools \
  --format json-pretty

This makes it possible to pair a reconstructed RIB or an update search with a contemporaneous ROA or ASPA view, rather than treating present-day authorization data as historical evidence. Together, remote execution, PCH parsing, reusable filters, corrected RIB reconstruction, and historical RPKI lookups support an investigation from source discovery through to a time-aligned routing and authorization result.

The release also adds --concurrency for local and server searches, improves bulk database refresh behavior, and updates to bgpkit-parser 0.18.0. TableDumpV2 RIB elements now expose a peer BGP Identifier when it is present, and known-but-unsupported path attributes remain available in the parser’s existing raw-attribute fields instead of being silently discarded. See the full v1.4.0 release on GitHub for the complete change list and release assets.