BIQ - Power Tools for the Data Analyst
Product Working With Us Articles Partners Company Downloads

White Papers


Perfect Cubes and Golden Chalices

It's compelling to think of data as going through a one-time process of cleansing and mapping, resulting in a "perfect spending cube" that represents Truth.

That's nonsense, of course. It's been our experience that, regardless of the time and money expended on a cleansing/mapping effort, an experienced commodity manager can drill into the tail of the distribution and almost immediately find spend that's poorly or incorrectly mapped. We've seen this sucker punch delivered many times to spend analysis vendors who claim to have "special knowledge" or "proprietary techniques" or "custom databases" that supposedly make them extra-special at cleansing and mapping.

Instead, if we admit that perfection is unattainable, this clears the air and lets us reach a reasonable solution. Since it is inevitable that we will find poorly-mapped
Why doesn't every vendor approach the mapping process in an incremental fashion? Because their rules systems are offline systems.
spend whenever we do a deep dive into a category, the BIQ philosophy is "fix it when you find it." Our mapping system operates in real time, right off the data viewer. Rules can be added and executed immediately, without any complex offline processes. Our collaboration feature allows multiple users to make changes in the rules and hierarchies, and coordinate those changes later.

So, BIQ users first do a straightforward and inexpensive overall mapping exercise (see Mapping Spend: Three Easy Steps). Then they examine the results, and focus more closely on categories of strong interest. For example, if Commercial Print is a relatively uncontrolled spending area, as it often is for financial services businesses, it's appropriate to dive in and create more intelligent mapping for the various types of Print spend (envelopes, lettershop, and so on). When that's done, it's time to move onto the next category. Errors are fixed as they're found, right then and there.

Why doesn't every vendor approach the mapping process in this common-sense, incremental fashion? Because their rules systems are offline systems. Their data cubes are read-only datasets. Offline procedures and mechanisms have to be brought into play for any changes. Then the dataset has to be "re-cooked" and "re-published." So these vendors must think about cleansing and mapping as a monolithic process, not as a series of incremental changes.

That's a losing proposition, and it gets worse with time. As offline changes are being made (typically when the data cube is refreshed, at the end of the month or quarter), more changes pile up at the door. It's the proverbial dog chasing the fire truck — he'll never catch up, and he falls farther and farther behind. As one BIQ customer put it, after converting from another spend analysis system, "With BIQ, we can get through many cycles of data clean-up before the next batch of data comes. We're finally ahead of the curve."