PM8: Hyper-Modeled Databases: A Crucial Technique for Truly Agile Data Warehousing

For decades now, data warehouses were extremely "brittle" in the face of new or changing business requirements. Once loaded, their data repositories could not be re-purposed without converting the existing data, a practice that can ruin expensive data conversion programming. As a result data warehouse designers were forced to doom their projects to "death by architecture," in which they insisted on perfect, up-front designs before data transform programming could begin.

Luckily, the new data modeling paradigms of hyper normalizing and hyper generalizing allow teams to readily adapt established warehouses in new directions without conversion script programming. Teams can now start with a simple design that meets current business requirements, then steadily evolve the database as new business needs emerge. These novel techniques have hundreds of successful implementations and strong support by vendors and practitioners. In this presentation, we will consider a couple of such case studies, then touch upon:

Why traditional data models become so expensive to update
Why "database refactoring" leaves much to be desired
Variations upon standard normal forms that allow agile data engineering
How highly generalized data models enable model-driven development
Where tools can amplify the power of these new modeling techniques

Ralph Hughes serves as Chief Systems Architect for Ceregenics, a Denver data analytics consulting firm. He has been building data warehouses since the mid-1980s, starting with mainframe computers, and has led numerous BI programs and projects for Fortune 500 companies in aerospace, telecom, government, and life sciences. He authored the industry's first book agile data management, and his third book, "Agile Data Warehousing for the Enterprise," was just released in October 2015. He is a faculty member at The Data Warehousing Institute, a certified Scrum Master, PMI Project Management Professional, and has coached over 1,200 BI professionals worldwide in the discipline of incremental and iterative delivery of large data management systems. He holds BA and MA degrees from Stanford University in computer modeling and econometric forecasting. He can be contacts at ralph.hughes@ceregenics.com or 303.274.9101.