700 words (8 minutes reading time) by Carole Parkinson

In many aspects of human endeavor, the belief that successive generations inherently improve holds true. Take sports, for example, where each Olympics sees records shattered by athletes who are better selected, trained, and conditioned than their predecessors. This continuous improvement is driven by a powerful incentive for each generation to surpass the achievements of the previous one.

However, this upward trajectory doesn’t always apply. In the realm of artificial intelligence (AI), there’s a phenomenon known as model collapse. Without diverse human-generated training data, AI systems risk malfunctioning if inundated with AI-generated content. The first generation of AI benefits from drawing on decades of human ingenuity, but as subsequent generations draw from their own created information, the pool of new ideas dwindles, resulting in a homogenised output.

Sudeep Srivastava, Co-Founder and Director at Appinventiv

Model collapse risk also exists in the conventional thinking apparent in Victorian local government. A colleague suggested to me that it has already happened. The sector is not responding quickly or effectively to changes in its environment. Since the major reforms of the 1990s, there has been a lack of sector-wide learning and knowledge transfer. Instead of developing unique models in the 1990s, the sector adopted private sector management practices. These practices have been perpetuated without proper consideration of their effectiveness.

The flattening of organisational structures that occurred at this time removed layers in the hierarchy that had provided opportunities for new leaders to take on responsibility in small steps with less risk and more time to learn. At the same time, vocational education moved to task specific training. The reductionist approach to workplace training, coupled with learning by rote, replaced holistic thinking and controlled experimentation to enable continuous improvement. Ongoing pressure to reduce costs impacted foundational ‘protections’ built into tasks, roles and responsibilities.

It’s fascinating how Karen Martin’s concept of isomorphic mimicry, as discussed in ‘The Outstanding Organisation,’ aligns with the idea of copying form rather than function. Attempting to mimic leaders can be akin to imitating the external appearance without truly understanding or replicating the underlying principles that make a leader successful. Copying form may create a surface-level resemblance, but it falls short in capturing the essential elements that shape an organisation’s leadership history, culture, knowledge, experience, and habits. 

Model collapse risk suggests that when leadership approaches are repeatedly grounded in the experiences of their predecessors they tend to emphasise common strategies and overlook unique situations. This tendency is analogous to leadership losing its long-term memory. With each iteration of leadership training based on historical experiences, existing biases within the decision-making framework can be magnified. Given that leadership outcomes often mirror the strategies employed, any biases within the decision-making process may become more pronounced over time.

As leadership continually learns from past experiences, there’s a discernible narrowing of generative capabilities. If a leadership team is influenced predominantly by its own interpretations of reality, there’s a risk of producing content that lacks diversity and neglects the representation of unique situations. This progression may result in a loss of originality within leadership approaches.

Finally, leadership encounters functional approximation errors when the strategies employed are not expressive enough. This becomes critical as these errors can compromise the accuracy of predictions, hinder technical understanding, impact organisational decisions, and potentially lead to financial losses.

Reflecting on a newspaper article from 2001 on careers in local government, it’s evident that leaders like Bob Seiffert, with 30 years of experience, were making significant strides. Manningham City Council was lauded for having more quality assurance qualifications than any local government globally. Seiffert was aggressively improving his organisation. Yet, it’s doubtful whether such achievements persist in today’s councils.

Unlike the past, where experienced local government leaders seamlessly transitioned to key roles in major public authorities, the trend has reversed. Former CEOs of the Police and Fire Services are now entering Victorian local government, signaling a potential regression rather than progression. Councils are no longer producing the public service leaders.

Cultural Evolution Society

This circumstances leading to model collapse are explained by Andrew McAfee in the “The Geek Way. He explores the mental model behind successful technology-based businesses, emphasising elements such as speed, ownership, science, and openness. He underscores the importance of cultural evolution in these organisations, challenging conventional thinking and highlighting the success achievable through a different mindset. It is worth reading.

The question now facing Victorian councils is whether they are capable of culturally evolving and developing a new mindset where each generation improves upon the last, or will they degenerate and a form of local government model collapse prompt further State government intervention. It’s time to critically evaluate our mental models and embrace a new way of thinking to ensure the continued growth and success of Victorian local government.

‘Council paths go in many directions’ by Carolyn Rance in The Age on 27 January, 2001

‘Preventing AI Model Collapse: Addressing the Inherent Risk of Synthetic Datasets‘ by Sudeep Srivastava, Co-Founder and Director at Appinventiv, November 2023

‘The Geek Way: The Radical Mindset that Drives Extraordinary Results‘ by Andrew McAfee, 2023

Footnote

I asked ChatGPT to explain model collapse to me, and this is what it told me:

Model collapse refers to a situation in machine learning, particularly in the context of generative models, where the model fails to capture the diversity and complexity of the training data. Instead, the model tends to produce outputs that are repetitive, lacking in variety, and often unrepresentative of the true distribution of the data.

In the context of generative adversarial networks (GANs) and other generative models, model collapse occurs when the model essentially ignores certain modes or features in the data and generates only a limited set of outputs. This can result in a loss of information and a failure to capture the full range of possibilities present in the training data.

Model collapse is a significant challenge because it undermines the goal of generative models, which is to create realistic and diverse samples that mimic the characteristics of the input data. When model collapse occurs, the generated content becomes less useful, less diverse, and may not accurately reflect the complexity of the real-world data the model was trained on.

Addressing model collapse often involves improving the architecture of the generative model, fine-tuning training parameters, or introducing regularization techniques to encourage diversity in the generated outputs. It is an ongoing area of research and development in machine learning to ensure that generative models produce high-quality and diverse samples.

I then asked ChatGPT how to detect model collapse.

Detecting model collapse in a generative model involves assessing the diversity and quality of the generated outputs. Here are several methods commonly used to identify signs of model collapse:

1. Visual Inspection:
   • Examine the generated samples visually. If the outputs consistently exhibit limited diversity, lack complexity, or appear repetitive, it could be indicative of model collapse.
   • Use visualization tools to compare the generated samples with the training data distribution and identify any discrepancies.
2. Diversity Metrics:
   • Employ diversity metrics to quantitatively measure the variety of generated samples. Metrics such as entropy, Inception Score, or Fréchet Inception Distance (FID) can provide insights into the diversity of the generated content.
   • Low values for these metrics may suggest that the model is not capturing the full spectrum of the training data.
3. Mode Dropping Analysis:
   • Mode dropping occurs when certain modes or features of the data are consistently ignored by the generative model. Analyze the generated samples to identify missing modes compared to the training data.
   • Techniques like kernel density estimation or clustering can help reveal the presence of missing modes.
4. Influence of Training Parameters:
   • Experiment with training parameters, such as learning rates or batch sizes, to observe their impact on the model’s behavior.
   • Changes in the model’s performance or the emergence of new modes may suggest sensitivity to specific training configurations.
5. Cross-Validation:
   • Split the training data into different subsets and train the model on each subset independently. Compare the generated outputs from each subset to identify variations or inconsistencies.
   • Cross-validation helps ensure that the model is not overly biased towards specific portions of the data.
6. User Feedback:
   • Incorporate user feedback to evaluate the perceived quality and diversity of the generated samples.
   • If users consistently report a lack of variety or unrealistic outputs, it may indicate model collapse.

It’s important to note that the detection of model collapse may require a combination of these methods, and there is no one-size-fits-all approach. Regular monitoring during training and thorough analysis of generated samples are essential for identifying and addressing model collapse in generative models.

Is there a method to detect model collapse as it happens in local government?