Category: Uncategorized

I’m Jim O’Neill. My high school art teacher fundamentally changed the way I see the world. He taught me that what we call lines don’t actually exist, only two planes of color that intersect and give the illusion of one. I’ve been looking for those intersections in data ever since.

I work as a quantitative systems architect applying the scientific method to infrastructure: forming hypotheses about system behavior, designing controlled experiments, building models, and forecasting cost and risk decisions that engineering instinct alone can’t make.

Over the past two decades I’ve done this at scale: a decade building workload analytics for some of the world’s largest MDM deployments at IBM, handling billions of records and trillions of comparisons across Fortune 50 and federal accounts. And now, characterizing the behavior of a global SaaS fleet — digging into API performance, modeling new product workload impacts, building finance-facing deployment projections, and using infrastructure telemetry in ways it was never designed to be used.

My career didn’t start this way. I used to dissolve moon rocks for a living. The discipline required to extract a signal at parts per quadrillion concentrations from a sample you can never replace translates surprisingly well to finding anomalies in a production system you also can’t afford to break.

No lines, and no second chances.

Those two experiences are why this blog exists; from entity and workload analytics to cost intelligence, anomaly detection, and what it looks like when you apply lab science thinking to complex systems.

My high school art teacher communicated in words and exhales. A humf over your shoulder carried more than most people’s sentences. One day while sketching a still life I heard a humf over my shoulder. I looked up. He looked down over the frame of his reading glasses and asked me a question I wasn’t prepared for.

“I see a lot of lines on your drawing. Look around the room. Point to one line you see in the world around you.”

We looked around the room. The edge of a desk. The frame of a window. The corner where two walls met. I surveyed. He waited.

“There are no lines,” he said. “There are two planes of shades of color that intersect and give the illusion of a line.”

I put my pencil down and looked at the drawing differently. Every edge I had rendered as a line was actually a boundary; the place where two planes of different shade or color met. The line wasn’t a thing. It was a perception. The real information was in the planes themselves: their relationship, their contrast, the quality of the boundary between them.

I have been finding those intersections ever since.

How a geochemistry lab at Notre Dame became the origin of everything I do in infrastructure today.

The question I get most often when people learn about my background is some version of: how did you end up here?

The résumé looks like a series of hard left turns; geology, then network consulting, then a startup, then a decade at IBM, then SaaS infrastructure. The only non-developer on a software architecture team. It doesn’t look like a straight line.

But it is one. It just requires the right lens to see it.

This post is a summary of my putzing around with a home built set of cheap low power nodes that I wanted to use as a 4 node hadoop cluster. My primary objective was to learn more about managing a hadoop cluster and to use the cluster for data analysis of some weather data I have been collecting for my frog locales. I wanted more experience with Python and R, so it seemed like this would be a good gear head project to take on. Continue reading

On perspective, pixel density, and why there is no spoon.

Most people open a log file looking for a specific thing. They find what they expected, or they don’t find it and conclude the answer isn’t there.

The log file isn’t the constraint. The question is.