Designing an Evidence-Based Audit Engine for IoT Security & Reliability

The Problem: audits are too slow, too manual, too inconsistent

IoT systems fail at boundaries: device at the edge -> gateway -> cloud ingestion -> processing -> dashboard. Each boundary introduces complexity in form of different protocols, certificates, networks and operational assumptions. That makes audits expensive and it’s a challenge to be consistent when evidence is collected manually.

In many traditional audits collection of evidence looks like:

• Screenshots.
• Tribal knowledge from a few software developers.
• Meetings.
• Recommendations that often are hard to reproduce.

That approach is not effective, when you need repeatability, traceability, and proof for internal governance or compliance such as NIS2 or Cyber Resilience kind of requirements.

I run technical IoT audit for clients to help identify gaps in reliability, security and observability. Checklists have been a great help here to streamline my audit process, but checklists still rely on a person to collect evidence correctly each time.

So I decided to build something better:
An audit engine that behaves like a command-line tool (CI).

Deterministic inputs -> automated evidence collection -> rule evaluation -> issues -> report -> sprint backlog

What I’m auditing (and why it matters in IoT/data pipelines)

When I do a technical architecture review, I map the IoT system end-to-end and look for risks across three fundamentals pillars:

• Reliability: failure modes, retries, queueing, backpressure, message loss, operational readiness.
• Security: transport security, identity & key management, secure defaults, hardening.
• Observability: tracing/metrics/logs, SLOs, incident debugging ability.

Where teams struggle varies a lot. Some teams are reliable but have weak security. Others are secure enough, but have a hard time with observability which becomes a challenge during incidents.

Example audit focus area: Transport security(TLS/mTLS)

Transport security sits at the boundaries where IoT systems tends to be be a bit messy.

For MQTT commonly used in IoT, a review quickly becomes detailed:

• TLS vs mTLS correctness
• protocol version
• cipher suites and compatibility
• certificate chain correctness
• certificate rotation strategy and expiry risk

Manually collecting all this evidence is doable, but it’s slow and error prone.

Design goals

I designed the audit engine around a simple idea:
Automate evidence collection and make results repeatable.

Concrete goals:
1. Repeatability: Same input and same output

2. Evidence-first: Store raw outputs, not just conclusions

3. Versioned rules: Rules evolve; over time, you should still reproduce past results

4. Measurable: Show improvement (or regression) between audit runs

5. Extensible probes: IoT spans many protocols and environments — probes should be pluggable.

6. CLI-friendly workflow: Command-line tooling that are predictable and scriptable.

High-level architecture

The high-level architecture consists of two main parts:

1. Rust probes (evidence collectors)

• accepts args (target, ports etc.)
• performs a specific check (e.g. MQTT TLS posture)
• returns structured JSON output using a shared contract

2. Scala ZIO backend (orchestrator + rules + storage)

• triggers audit runs for assets
• executes probes and captures raw JSON outputs as evidence
• evaluates rules against evidence to generate issues
• stores evidence/issues/audit runs in a database
• produces a report

Rust probes: why Rust, what a probe looks like, and what it outputs

I chose Rust because it’s excellent for building fast, reliable CLI tooling and it scales well from low-level details to higher-level ergonomics.

A probe is a simple binary:

• parse CLI args
• connect (example: tokio::net::TcpStream)
• wrap with tokio_rustls::TlsConnector
• collect findings into a typed output struct
• serialize to JSON

Here is an example output contract:

#[derive(Debug, serde::Serialize)]
pub struct ProbeOutput {
    pub probe: &'static str,
    pub version: &'static str,
    pub status: &'static str,
    pub summary: String,
    pub started_at: String,
    pub findings: Option<TlsFindings>,
    pub error: Option<ProbeErrorView>,
}

Scala ZIO backend

The backend consist of a HTTP API with an auditEngine controller that includes an endpoint:

POST /assets/{assetId}/audit-runs

Flow:

1. First an asset is fetched from database through a repository
2. create an auditRun record
3. run one or more probes via CLI execution (ZIO Process library)
4. store raw JSON output as Evidence
5. evaluate rules against evidence
6. upsert generated issues and complete the audit run

override def runAuditForAsset(assetId: Long): Task[AuditRunResponse] =
    for {
      asset <- assetRepo
        .getById(assetId)
        .someOrFail(new RuntimeException(s"Cannot get asset to run audit: $assetId doesn't exists"))

      now <- zio.Clock.instant

      auditRun <- auditRunRepo.create(
        AuditRun(
          id = -1L,
          assetId = asset.id,
          status = AuditRunStatus.Running,
          startedAt = now,
          finishedAt = None,
          issuesFound = 0
        )
      )

      probeJson <- probeRunner.runMqttTlsPosture(asset)

      evidence <- evidenceRepo.create(
        Evidence(
          id = -1L, // generated by the DB,
          assetId = asset.id,
          probe = config.probe_name,
          collectedAt = now,
          rawJson = probeJson,
          auditRunId = auditRun.id
        )
      )

      issueDrafts = rules.evaluate(evidence, asset)

      issues <- ZIO.foreach(issueDrafts)(draft =>
        issueRepo.upsertFromDraft(asset.id, draft, evidence.id, auditRun.id, evidence.collectedAt)
      )

      _ <- auditRunRepo.complete(auditRun.id, issues.size, now).ignore

    } yield AuditRunResponse(auditRun.id, asset.id, issues)

A rules engine is used to evaluate evidence by using JSON parsing, normalization and pattern matching to generate potential issueDrafts like deprecated TLS protocol version.

IssueDraft(
 key = "mqtt_tls.outdated_tls_version",
 severity = "High",
 title = "Outdated TLS protocol version in use",
 description =
  s"The MQTT endpoint negotiates an outdated TLS protocol version ($tlsVersion). " +
  "TLS 1.0 and 1.1 are deprecated and vulnerable to known cryptographic attacks. " +
  "Only TLS 1.2 or TLS 1.3 should be allowed",
)

How I use this in real client engagements

In client audits, I use the engine to accelerate and standardize the parts that should be automated:

• collecting evidence
• turning evidence into a prioritized backlog of issues
• rerunning audits to verify improvements after changes

That gives two benefits:

1. Less time spent collecting evidence manually

2. More time spent on expert judgment — architecture tradeoffs, risk prioritization, and operational hardening.

It’s still early-stage, but it already helps me produce:

• consistent evidence packages
• faster iteration across environments
• measurable posture changes across runs

Next steps

If you’re building or operating an IoT system and want a clear view of gaps across security, reliability, and observability, I offer a deep technical IoT Audit.

What you get:

• evidence-based findings (not vague recommendations)
• a prioritized backlog
• clear next steps for hardening production

Feel free to contact me.