Trough of Disillusionment and the Pythagorean Cup

Disillusion can become itself an illusion If we rest in it.

T. S. Elliot

This post extends the application of the Hype Cycle to an organization’s data and analytics initiatives by integrating a data maturity model into it. It also visualizes the drastic onset of negative hype using pythagorean cup.

Hype cycle, data analytics, maturity model

The Hype Cycle for an organization’s Data & Analytics Initiatives
Adapted from Gartner’s Hype Cycle and Data & Analytics Maturity Model

Hype Cycle

Introduced in 1995, the hype cycle has gained popularity in the media as a tool to discuss technological innovations. Gartner suggests that it can be used to identify overhyped areas against those that are high impact, estimate how long technologies and trends will take to reach maturity, and help organizations decide when to invest/adopt.

While I am not convinced of the prescriptive or even predictive abilities of the model, I do appreciate the underlying notion of how humans interact with newness. I think the simplicity and transferability make it a good addition to one’s critical thinking toolkit. In a world overfeeding on information, it is a good reminder of the relationship between hype and maturity. 

The hype cycle is a combination of two curves, shown in the figure below.

The hype-driven expectations curve reflects public speculation and excitement that comes in a rush, peaks and drops. Maturity refers to the actual development and application, which increases gradually.

The 5 Stages of Hype Cycle

  1. Technology Trigger: An initial event attracts significant interest in an innovation.
  2. Peak of Inflated Expectations: Feedback from early adopters and initial successes, results in  over-enthusiasm and unrealistic projections.
  3. Trough of Disillusionment: Decrease in interest because the innovation does not live up to its overinflated expectations, 
  4. Slope of Enlightenment: Focused experimentation and solid hard work by a dedicated group reveals the true potential and benefit of the innovation. 
  5. Plateau of Productivity: The real-world benefits of the innovation are demonstrated and accepted. 
Hype cycle, trough of disillusionment, peak of inflated expectations

Source

Hype Cycle for Data & Analytics  (D&A)

Over the years Gartner has released 100s of hype cycles for various markets, including one for data science. However, my goal is to extend the application of this simple curve to the D&A efforts of an organization (not a market).

The figure below combines the hype-driven expectations curve with the Gartner D&A Maturity Model.  There are quite a few maturity models out there but most seem to confuse levels of maturity with lifecycle of a data science project. These models suggest that the process of going from raw data to prescriptive analytics reflects data maturity.

The model I chose combines “business outcomes, people, skills, processes, data and technologies”.  It also just happens to have 5 levels that complement the inflections in the hype cycle:

  • Level 1: Basic
  • Level 2: Opportunistic
  • Level 3 Systematic
  • Level 4: Differentiating
  • Level 5: Transformative
Hype cycle, data analytics, maturity model

Hype Cycle for an Organization’s Data & Analytics Initiatives
Adapted from Gartner’s Hype Cycle and Data & Analytics Maturity Model

Hype Cycle and Pythagorean Cup

Now, that we have our hype cycle, let us explore it further using the Pythagorean Cup.

The pythagorean cup is a container with a central column in it. When filled beyond a certain point, the cup is drained completely of its contents because of a siphoning effect. As long as the level of the liquid does not rise beyond the level of the column, the cup holds its contents as normal. Learn more here.

Hashemi et al, use this ingenious device to explain the dynamics of the hype cycle. The liquid represents expectations in the figure below. So, the first three panels, correspond to the rising expectations until they reach the Peak of Inflated Expectations. In the two right-most panels, the liquid is draining out of the glass representing the negative hype until it dies down in the Trough of Disillusionment. 

The big takeaway from this model is that there exists a critical point on the hype cycle which marks the onset of negative hype leading to the Trough of Disillusionment. If organizations, can keep the expectations from reaching that critical point, the drain can be prevented!

Source

Eliminate the Trough of Disillusionment

To make sure that an organization’s D&A efforts don’t die off, there are two approaches to mitigate, if not eliminate, the trough. :

  • Manage expectations: By setting realistic expectations (identifying specific goals and barriers) can lower the peak of inflated expectations. The cup won’t fill as fast.
  • Accelerate maturity: By inducing an early inflection in the maturity curve, the trough will be shallower. The cup won’t drain as fast.
Eliminate the Trough of Disillusionment by accelerating maturity and managing expectations

Eliminate the Trough of Disillusionment by accelerating maturity and managing expectations

Accelerate maturity

To accelerate D&A maturity such that the organization is at least a Level 2 before the expectations peak:

  • Assess the existing infrastructure and information assets.
  • Start working towards a standardized way to work with data. 
  • Identify areas of greatest business impact in the organization. Focus on business drivers and outcome measures. Success of these initiatives will bring visibility to the D&A efforts.  
  • Leverage the existing skillset by bringing together D&A enthusiasts from across the organization.  Invest in their professional development. Encourage mentorship and collaboration. Promote data literacy.
  • Implement agile work principles. Aim for minimum viable products and short delivery cycles

Manage expectations

The D&A leaders should be mindful when communicating stakeholders in early phases. It can be tempting to let people dream that data is the answer to all their problems, but it is crucial to close the gap between expectations and real capabilities. 

  • Assess whether you are exploring data just because it is available or because it addresses a specific and actionable business question. Ask the right business questions.
  • Be transparent about the quality and limitations of the data. Clearly communicate any gaps that are found. Focus on data libation and reliability.
  • Communicate the cost vs opportunity of the proposed data projects. Identify barriers to success. Define measures of success to assess ROI for D&A efforts.

Summary

The most popular use of the hype cycle is to give a snapshot of relative hype & maturity of technologies within a certain segment of the IT world. This post extended its application to an organization’s data and analytics initiatives.

Inserting a specific maturity model into the cycle built a relationship between various levels and inflection points on the underlying curves. This comprehensive approach can be used to guide the D&A strategy.

Picturing the 5 stages as a pythagorean cup, helped visualize the drastic onset of negative hype.  It also highlighted the importance of not letting the expectations peak too high.

Putting all together, helped identify two approaches to eliminate the trough of disillusionment: Manage expectations and Accelerate maturity. This mental model can now be applied to any initiative/innovation at any level.