A Preliminary Model of Organizational Complexity: Optimizing Chaos in Organizations

Readings in the business and the research literature show a progression of thinking in organizational systems, from the structural (team development) to process-oriented (organizational learning). Often similar areas of focus emerge from different writers at the same relative time - Drucker (1992) and Argyris (1992) both deal with organizational learning, as well as organizational design issues. Teams emerged as a hot topic from 1993 (Katzenbach and Smith) to 1996 (Donnellon). In my literature review, only Banathy (1996) deals strictly with social systems, and he also presents complexity theory as an effective model for understanding organizational systems. Complexity theory also emerges as the broadest context from which to approach social and organizational systems. 
 

Some of the dimensions identified in this model are consistent with complexity theory: entropy vs. chaos, and the degree of structure imposed on the organization. The proposal inherent in this model is that an optimized level of complexity and order can be reached, although no prescription can be inferred from the model. The basic elements explored here consist of:

Structure essentially the degree of order imposed on the organization, structure is a variable relative to cultural and social norms. However, structure can be described as the system of rules, levels of hierarchy, fixed roles, and separate compartments within an organization. Structure comes with a cost - it requires energy and overhead within the organization just to maintain it. Also, too much structure leads to entropy, in the organizational/social sense - members forced to work within the confines of a highly structured, rule-bound organization constrain their contributions to adapt to the formal structure. While any creative individual will find ways to work around the confining system, reward processes in these organizations tend to keep those that closely follow the structure in positions of authority, which perpetuates the process. 

The lack of any structure can lead to problems in the organization also. Without the guidelines of useful process, members can work within self-organized groups that defeat or sub-optimize the overall institution. Although goals, leadership, and direction can produce an environment where different teams support the overall organization, in practice small groups become unfocused and disconnected with the larger organization. Too little structure can lead to chaotic conditions, where individuals lose track of the "big picture" of projects and initiatives, attending to their local work practice. 

So there is a theoretical balance between too much structure and too little. Groups within the larger organization should be free to seek some measure of self-organization, which makes the most of natural patterns of interest, energy, and skill. As self-organization tends toward the chaotic, it might be constrained by larger formal structures, processes, and goals that define its boundaries and scope, but do not manage through hierarchy. This can allow for local optimization of the workgroups for projects and initiatives, while allowing managers to maintain resource projections and availability for the needs of the larger organization. 

This optimal area of structure/order can be considered an "affordable complexity", a state wherein the organization accepts a fair amount of "disorder", especially in rules and hierarchy, to free up resources and energy for the dynamic state required for creative response to changing conditions affecting the organization. Not only will this state maximum the creativity within workgroups due to the synergy of team commitment, but synergy among groups can also be coordinated in the same way. This optimized state of organizational energy has the capacity to produce extraordinary productivity, creative breakthroughs, and personal satisfaction within the organizational setting. Obviously this optimal complexity cannot be produced by mandating it from above, but it can be fostered by those in authority. Like any evolutionary process, though, the organizational ecology must be initiated with appropriate conditions, then allowed to progress and cycle into a state of optimal complexity. It cannot be "designed" in the classical sense - in order to enable creative self-organizing, managers would have to construct the best starting conditions, roles, and resources, and then let the natural complexity processes take over. Reorganization, new structures, or role changes added to the process could be devastating, however - in a complex ecology, very small influences can have far-reaching and quite unpredictable effects. Managers roles would change significantly, requiring them to adopt a more nurturing, supportive approach, more like farming than hunting. Control-oriented styles or micromanaging would be the worst possible interventions at the local level, and large-scale reorganization could be disastrous to the formation of commitments, alliances, and productive workgroups across the larger organization. In this ecology, individuals would have to be trusted implicitly by those in charge - trusted in their ability and commitment. Managers should also realize that in this model, those not delivering on promises would not be protected by their association with more powerful allies - teams would merely exclude those that don’t produce or cooperate, and "dissociated" members would eventually fall from the organization.

Commitment, or Energy - This dimension can also described in broad terms, as part of the model. Within most corporate organizations, contributions of individuals result from performance in assigned roles. People do the job asked of them, and perform at a level of energy and commitment typically commensurate with expectations. Although this is an extremely variable dimension, in general, people do not outwork their set expectation. However, when this dimension is evaluated within the complexity model, a different perspective becomes available. The "chaotic attractor" function can significantly drive up creative energy, and can lead to breakthroughs in ideation, design quality, and productivity. 

Basically, opportunities emerging within the work context serve as both chaotic attractors - leading to bifurcations. Opportunities are highly situational, difficult to observe or predict. In this context I define opportunity as the emergence of conditions recognizable to interested and motivated actors that afford the possibility for fulfillment in the direction of the work interest. In other words, people look for conditions that serve the progression of their specific personal and professional interest. In a self-organizing workgroup, members would be free to act upon emerging opportunities whenever an opportunity held promise. Yet, in most situations, the normal workload must also be maintained. The extra energy made available in the commitment to seek the opportunity is driven by this context. 

This phenomenon explains how I was able to, in 1996, change jobs to a challenging new position in a new company, complete a large professional book for a McGraw-Hill contract, pursue my PhD program with the Union Institute, and pursue other opportunities as they emerged. Energy follows interest, which is how opportunities seem to show up as chaotic attractors, or more specifically, as bifurcations resulting from the chaotic attractor. Within the context of interest and involvement in an organization, the opportunity is itself a bifurcation. As Ben Davis puts it, "emergence and attraction lead to ‘found goals’," much like improvisation in jazz. To the one who first recognizes an opportunity within their own context, it looks improvised, or "made up." The "first bifurcation" or a new opportunity has no "audience," to continue with Davis’ jazz example. It must become its own force, and grow into a new stable state. This model could be drawn out much further, and can perhaps be explored more in the Systems Theory and Social Systems Design work that follows from this exploration.

Finally, the two dimensions identified here - Structure and Commitment - are not sufficient for explanation of organizations, but they are useful distinctions that add to understanding complexity in organizational systems. Commitment level is not the only driving factor for complexity against structure, but one of probably several. A pure organizational design model might evaluate structure vs. process (business process, work practice complexity, and policy) but I consider that generally process attributes are implied in structure (more structure = more process). Product complexity also interacts with structure, with inherently more complex products demanding higher and optimal complexity from the organization.

There are other useful interactions that arise between the two dimensions. One is that organizations can change structure, but only individuals can supply commitment. Organizations (through leaders and representatives) can ask for commitment, or sustained energy, but only the individual member of the organization can offer this quality. 

Background

Interestingly, although this topic seems to be a natural for application to organizational studies, there were not many recent citations from current literature. After developing my model, I decided to validate my thinking by searching for other references. Through an online term search ("complexity theory" and organization and system), I retrieved only several relevant articles, and this concise description from the Organizational Issues Clearinghouse, published by Professor Philip Anderson, Amos Tuck School, Dartmouth College (1996):

Complex organizations have been important in organizational studies for decades. Historically, scholars have examined vertical complexity (number of hierarchy levels), horizontal complexity (the number of differentiated departments), and spatial complexity (the geographic dispersion of organizational subunits). Organizational environments have also been characterized as more or less complex depending on how heterogeneous and dispersed resources are within them.

However, a different view of complexity is emerging that may have important implications for organizational scholarship. The study of complex system dynamics has perhaps progressed farthest in the natural sciences, but it is also beginning to penetrate the social sciences. This interdisciplinary field of study is still pre-paradigmatic, and it embraces a wide variety of approaches. Although it is not yet clear whether a genuine science of complexity will emerge, it does seem clear that scholars in a variety of fields are viewing complexity in a different way than organizational scholars traditionally have.

A number of findings now seem fairly well-established, including the following:

 
• Many dynamic systems do not reach an equilibrium (either a fixed point or a cyclical equilibrium).
• Processes that appear to be random may actually be chaotic, in other words may revolve around identifiable types of "attractors." Tests exist that can detect whether apparently random processes are in fact chaotic.
• Two entities with very similar initial states can follow radically divergent paths over time. The behavior of complex processes can be quite sensitive to small differences in initial conditions. This can lead to highly path-dependent behavior, and historical accidents may "tip" outcomes strongly in a particular direction.
• Very complex patterns can arise from the interaction of agents following relatively simple rules. These patterns are "emergent" in the sense that new properties appear at each level in a hierarchy.
• Complex systems may resist reductionist analyses. In other words, it may not be possible to describe some systems simply by holding some of their subsystems constant in order to study other subsystems.
• Time series that appear to be random walks may actually be fractals with self-reinforcing trends. In such cases we may observe a "hand of the past" in operation.
• Complex systems may tend to exhibit "self-organizing" behavior. Starting in a random state, they may naturally evolve toward order instead of disorder.

Argyris, Chris. (1992). On organizational learning. Cambridge, Mass: Blackwell Publishers, Inc.

Banathy, B. (1996). Designing social systems in a changing world, Plenum Press, New York.

Donnellon, Anne. 1996. Team talk. Boston: Harvard Business School Press.

Drucker, Peter F. 1992. The new society of organizations. Harvard Business Review Sept-Oct.:95-104.

Katzenbach, J. R., and Smith, D. K., (1992). The Wisdom of Teams, Harvard Business School Press, Boston, Massachusetts.