Data Engineering | texlytics

Illustrated infographic titled “The Most Expensive AI Model Is the One You Didn’t Need” by J. Michael Smith, showing how an AI request is routed by intent, confidence, and policy to deterministic data retrieval, a small specialized model, or a complex reasoning model based on the intelligence required.

25/09/ 2026

The Most Expensive AI Model Is the One You Didn’t Need

There is a growing conversation about the cost of artificial intelligence. Tokens cost money. GPUs cost money. AI subscriptions cost money. Enterprise licensing costs money. As organizations put AI into more business processes, those costs scale surprisingly quickly. All of that is true, and I think most of the conversation is happening at the wrong layer. Before asking what an AI model costs, we should ask two simpler questions: Did this task need an AI model at all, and if it did, did it need that model? That…

11/09/ 2026

We Built the Grader Before We Built the Agent

Anyone can build a chat window that answers questions about invoices. Connect a language model to the data, ask, “What is our outstanding AR for this company?” and it’ll give you something. In a demonstration, it can look like magic. The problem is the word something. In accounts receivable, an answer that is close isn’t a smaller version of the right answer. It’s a wrong answer wearing a suit. If someone carries that number into a collections call, financial report, or board presentation, “approximately right” isn’t helpful. It’s…

04/09/ 2026

Your ERP Is Not Your Source of Truth. And That’s Okay.

One of the most common questions I hear when talking with business leaders is, “What’s the best ERP system?” It is a reasonable question, but after more than twenty-five years of designing integrations, building data platforms, and helping organizations solve technology problems, I have come to believe it is the wrong place to start. The better question is not which system can do everything. It is how we make all the systems already running the business work together. The Reality of Modern Business I have yet to work…

07/08/ 2026

Your Warehouse Is the Index the API Won’t Give You

A pull I had run cleanly for months started dying. Same endpoint, same code, same credentials. It would page through tens of thousands of AP registered invoices and then, somewhere north of 19,000 records, fall over. The frustrating part was the consistency. It did not fail randomly. It failed at the same place every time. That consistency turned out to be the whole story. By the time I was done, the ceiling had moved on me twice, and the most dangerous version of the failure was the one…

24/07/ 2026

The Dedupe Key That Doesn’t Drift

This article was originally published on LinkedIn and is archived here on texlytics.com. There is a reflex every data engineer has when they need to deduplicate rows: hash the row, compare the hashes, keep one of each. It is fast, it is simple, and on a well-behaved source it is correct. I reached for it without thinking when I built the ingestion for one of our CMiC AP voucher endpoints. It was the wrong call, and the way it failed is worth writing down, because the lesson holds…

10/07/ 2026

The Most Important Table in My Microsoft Fabric Environment

This article was originally published on LinkedIn and is archived here on texlytics.com. Most data engineers love talking about pipelines. Others love talking about notebooks, Spark, semantic models, or dashboards. But after building a near-real-time Microsoft Fabric integration framework that ingests dozens of CMiC endpoints every fifteen minutes, I’ve come to a different conclusion: The most important object in my entire environment isn’t a pipeline. It’s a table. Not a fact table.Not a dimension table.A control table. What started as a simple metadata repository evolved into the operational…

03/07/ 2026

We spent a year building a data platform. Here’s what AI can finally do with it.

This article was originally published on LinkedIn and is archived here on texlytics.com. Most construction firms that say they want “AI” don’t have data that AI can use. The accounting system doesn’t talk to the project management tool, half the operational truth lives in spreadsheets, and the same vendor is spelled three ways across three systems. Point the fanciest model on the market at that, and it will confidently produce nonsense — because it’s learning from a mess. So before any of what follows, the boring part. We…

26/06/ 2026

Stop Writing One Pipeline Per Source

This article was originally published on LinkedIn and is archived here on texlytics.com. Every data engineer hits this moment. You’ve integrated three systems, you’re proud of it, and then a fourth source shows up. So you do what you did the last three times: copy the previous notebook, find-and-replace the table names, tweak the date logic, adjust the pagination, ship it. It works. You now have four notebooks that are 80% identical and 20% subtly different — and that 20% is where every future bug will live. Then…

19/06/ 2026

The Bug That Reported Success

This article was originally published on LinkedIn and is archived here on texlytics.com. One of our customers didn’t exist. Not deleted, not archived — simply absent. A real company, with real invoices, that should have been sitting in our analytics layer like everyone else. It wasn’t. And the system responsible for bringing it in had run cleanly every night for weeks. Green checkmark after green checkmark. No errors, no warnings, no failed jobs. The whole time, it was quietly wrong. This is a story about why the bugs…

12/06/ 2026

Our ERP’s API returned different results for the same query. Here’s how we dealt with it.

This article was originally published on LinkedIn and is archived here on texlytics.com. Run the query at 9:00 AM. You get 4,812 rows. Run the exact same query at 9:01 AM. You get 4,797. Run it a third time and you get 4,812 again — the exact count you started with. Relief. Except that run is the most dangerous of the three, and I’ll come back to why. No data changed. No filter changed. Same endpoint, same parameters, same everything. Fifteen rows just evaporated, and they were different…

Category Data Engineering