Clayco · $8B design-build firm · 2023–2025
Receivables forecasting
Hundreds of millions in open invoices scored daily, median error 13 days
$100Ms
Forecast daily
open invoices, 30-day window
13 days
Median error
tens of thousands of closed invoices backtested
1M+
Predictions served
daily batch runs since Mar 2024
2.5 yrs
Still running
through a handoff, after I left
forecast error, dollar-weightedmean absolute, by invoice value
forecast error, medianthe typical invoice
The state before
A design-build contractor bills on progress, not on delivery. Cash is committed long before it arrives, and the gap between the two is where corporate treasury lives. Planning against that gap meant forecasting when open invoices would actually pay, and that forecast was assembled by hand each cycle: net terms, customer history, and an experienced analyst’s sense of which accounts ran slow.
It cost hours of skilled finance time on every refresh. The larger cost was that the number could not be interrogated. It arrived as a single figure with a person’s judgment behind it, so nobody downstream could ask what the next thirty days looked like if the largest invoices slipped, or which part of the business was carrying the exposure.
Why the obvious approach fails
The obvious framing is regression: take closed invoices, learn days-to-collect, minimize error. It breaks in two places.
The first is censoring. At any moment most invoices in the book are still open, and the ones that have closed are not a fair sample of the ones that haven’t. Fast payers close first. A model trained only on completed invoices learns from a population biased toward the behavior you least need predicted. This is the exact condition survival analysis was built for: it uses the open invoices as evidence too, by treating “still unpaid at day 40” as information rather than as a missing label.
The second is the shape of the answer. The treasury does not need a date per invoice. It needs dollars in a window: how much lands inside thirty days, how much between thirty-one and sixty, and what the probability is that a given invoice runs late. A point estimate cannot answer that. The output had to be a distribution before it could be useful, regardless of how accurate the point was.
Then there is the noise floor. Collection times on progress-billed work swing from same-day to past six months, and the spread is wider than the average wait. Any single-number accuracy claim has to be read against that, which is why the metric I built the monitoring around is dollar-weighted rather than raw: being ten days wrong on a small invoice and ten days wrong on one two orders of magnitude larger are not the same error.
What I built
An ensemble of two models with different jobs. A gradient-boosted regressor sharpens the central estimate. A survival model produces the hazard curve, and from it the probability-late figure and the dollar decompositions the treasury actually consumes. The final prediction is the mean of the two, which held up better across slices than either alone.
Validation was not a single holdout. I benchmarked against the prior manual approach and against simple heuristics, net terms and per-customer historical means, because a model that cannot beat the rule of thumb is not worth operating. Then I sliced: new projects, new customers, per subsidiary. The failure mode that matters is a job with no history, since that is precisely where a person’s intuition has the least to go on and where the model was supposed to add something.
The second half of the work was making it survive. The pipelines began on a managed platform with visual recipes and scheduled scenarios, which is a good place to build and a bad place to own. I containerized the whole thing: Docker images pushed to ECR, scheduled ECS Fargate tasks with separate entrypoints for the training and closed-invoice pipelines, Compose for local parity, scoped IAM task-execution roles, credentials out of the code and into Secrets Manager, logs into CloudWatch. I wrote the team’s internal docs for that setup, because the migration was only worth doing if the next person could operate it.
Monitoring was part of the deliverable, not an afterthought. Rolling amount-weighted MAE, collection mean and standard deviation, the ratio between the two, and automated alerts when the ratio drifts. The ratio is the honest signal: it asks whether the model is still beating the natural variability of the thing it predicts, which is a question a raw error number quietly refuses to answer.
What changed
The system has scored tens of thousands of distinct invoices since its first prediction in March 2024, across more than a million prediction rows, and it still runs every day. The current batch covers a couple of thousand open invoices and hundreds of millions of dollars of forecast collection inside thirty days. Backtested first-prediction-versus-actual on tens of thousands of closed invoices, median absolute error is 13.2 days and dollar-weighted error is 19.8 days, well inside the natural spread in when invoices pay.
Finance dashboards consume it directly for corporate cash-flow planning. The forecast stopped being a number a person produced and became a surface other people could question.
What I take from it
The model was not the hard part. Choosing an output the treasury could act on, and proving the thing against the incumbent process rather than against a validation split, is what made it get used. Containerizing it is what made it still be running two and a half years later, through a handoff to another engineer and long after I left the building.