"Measuring the Unmeasurable: Markov Chain Reliability for LLM Agents," by Phat Tuan Tran-Truong and Dr. Xuan-Bach Le, has been accepted into the Research Track of ISSRE 2026 (IEEE International Symposium on Software Reliability Engineering), CORE Rank A, held in Limassol, Cyprus, October 2026. LLM agents increasingly behave like sequential software systems, yet their reliability is usually summarised by a handful of scalar benchmark numbers. Metrics such as pass@k, pass^k and the reliability decay curve (RDC) are useful summaries, but they do not identify which success-time distribution is being estimated, test whether the traces support that distribution, or quantify the uncertainty that comes with a finite number of traces. TraceToChain is a reproducible pipeline that fits execution traces to an absorbing discrete-time Markov chain with explicit diagnostics and uncertainty: it builds an automatic cluster taxonomy, estimates transitions with Laplace-smoothed maximum-likelihood estimation, checks goodness of fit with a composite AIC and Kolmogorov–Smirnov certificate, and reports both Dirichlet-posterior credible intervals and non-parametric bootstrap intervals. Across seven controlled MAST-style frameworks under a strict 50/50 fit/test protocol, held-out empirical RDCs overlay their analytic counterparts with a maximum L∞ of 0.053. A two-sample KS test accepts the fitted chain with p > 0.05 on all seven frameworks.