In this work Phin Upham takes us through Jeffrey Pfeffer’s provocative propositions on theories of organizations. Are theories of organizations initially observation-driven, do they lose their connection to reality over time?
Most good theories, if not all, in OT have their foundation in observation and reflection – no great insight spring straight from phenomena. Great theorists like Burt or Selznick are masters at studying something closely and then making some general statement that picks up from this observation but says something about other firms or organizations. Selznick, for example, in his book on the TVA studies a specific governmental institution carefully, provided us details on what he say, and drew from this universal understandings about organizational recalcitrance, agency problems under lax control, the public sector, etc. No one is interested in reading a close study of an organization, no matter how well written, if the author is simply going to list the things he saw there. What we are looking for is a SELECTION of his observations (he was there a year, he does not report everything) and what we want is a selection that supports some larger point. We want a scholar to use his research as a grounding for theories and propositions that he can substantiate based on his observations. Emerson once said as a word of advice to schoolchildren [and future academics] “it is fine to build castles in the air, but one must first build ones foundations on the ground.” Similarly, theories are fine as long as they are grounded in phenomena.
Selznick’s study begins with the problem the organization is encountering – to revive an area of the country through encouraging its power and production of fertilizer. This mandate, broader than actual powers, left much room for action and coordination of action that was beyond the scope of most governmental organizations. The TVA was seen to be very successful at the time, but Selznick looks at it closely, using personal experiences and also organizational literature to ground his observations. He discusses the unanticipated consequences in detail. Only with this foundation does he discuss what he sees are the principles that these substantiate. So, for example, the idea that organizations without clear goals and with too much autonomy will tend to coopt power and lack control. He grounds his ideas in specific examples and generates then from specific phenomena. If one wished to disagree with him, one could study another governmental structure which had similar characteristics, or even go back and reinterpreted the TVA. A work need not be a case study to be phenomena driven. Specific propositions and a careful study often help to do this as well. Portes in “The Informal Economy and its Paradoxes” successfully studies the phenomena of informal markets, generating theory without overgeneralizing. Testability and verifiability are crucial.
The temptation, of course, is to take an idea as far as it can go without the substantiation. This has proven successful for some scholars. Transaction cost theory, for example, is largely theoretical. Our reading this semester of Markets and Hierarchies included little direct observation and a lot of theory and extrapolation. Nancy Cartwright, a philosopher of science at Oxford (or Cambridge) has argues that no observations have causality or can be used to draw causal connections. She says that only through simplifying assumptions, through the creation of a false but useful model of the world can causality be brought into a theory. Thus, every proposition, every theory, contains assumptions and contains interpretations. Thus, in this construction, while no amount of observations creates or, ultimately, justifies a theory, similarly, a theory without observation or one too far removes from reality looses its causal power. A tight fit between reality and theory is a necessary think but this fit can never be perfect. Some great theorists, like Weber, have gotten away with largely hypothetical theorizing, but most do not.
Some authors draw their theory from observation, and whose propositions were closely, though not perfectly, a result of their research. Perhaps the best example is the previously discussed Selznick in his book on the TVA. Since it was a case study, he was able to see what was going on and create theories to model the happenings. His theory about organizational recalcitrance, for example, came directly from his observations of organizational inflexibility over time in the TVA. Similarly, but suing opposite methodological tools, Carroll and Hannan based their statements about organizations carefully on their birth-death model of firms. For the populations they study their results seem to come out clearly. It does seem true, for example, that there is a liability to youth and smallness in as far as young and small organizations die more frequently. But to go beyond this and make some statement as to WHY this is true on an organizational level would go beyond the ken of the theory. So this is an example of a methodology which is very interesting and raises interesting questions but does not allow for causal extensions to go very far. Since their analysis is at industry level rather than the firm leave anyway, they tend to avoid theories about firm action, preferring to speak about industry behavior. Notice, though, that in order to generate this industry level theory they had key assumptions such as that firms do not change much, etc. Another example an essay close to their subjects as the another case study read – Dibble’s analysis of English County Governments which studies the transfer of power between central and local control. This is not to say that all case studies draw conclusions that are ties to their observations. Crozier’s study of the French Culture, though fascinating, must have been largely speculation and would certainly meet many doubters.
One of the central questions with phenomena driven vs. theory driven cases is one of precedence. Do you have an idea and search for it in firms (one or many) or does one observe firms and then generate ideas from them. Selznick, for example, seemed o observe organizational recalcitrance before he thought of the idea, but Carroll et al. often seemed to be testing specific hypothesis on their data. They often extend their ideas to include all organizations even though the types of industries they study are specific. But the problem is that such cause effect is not clear. These are scholars of organizations, so past contact with organizations can generate ideas which they then test on another set of organizations. So the structure of the paper in this sense is less helpful. If a theory springs from observation of certain phenomena, and it is then generalizes, it ought to be tested elsewhere before it is accepted, just to be sure. If a theory was generated from the literature and then an appropriate place was selected to test it (as with Bert) then this is more convincing. In statistics, this is seen as the difference between a one sided and a two sided test. If you are drawing from the data, you need a higher level of significance.
Meyer and Rowan’s piece on Myth and Ceremony and DiMaggio and Powell on the Iron Cage, for example, both use their knowledge of organizations to draw specific hypothesizes which are both testable and falsifiable, though they do not test them here. Further, they are concerned with the extend to which organizations are social and interact socially. They ask and hypothesize about he effects of their theoretical findings. The three essays we read in Economic Sociology, particularly the one on the informal economy, are wonderful examples of theories based on observation and reflection but still firmly tied into real questions that the authors asked. Other people reach similar conclusions but using vastly different methods but are fundamentally interested in similar questions. For example, Thompson largely uses logical reasoning in this essay while Lawrence and Lorsh use a much more quantitative way of testing theories but both are interested in how organizations organize themselves given their environment. Similarly, Pfeffer/Salancik use theory with a dab or statistics to illustrate, while Burt sets out his propositions and tests them through directorship ties but both are interests in the way that firms use and combine scarce or incomplete resources.
Other theories, arguable, have gone beyond their initial phenomena and taken on broader claims. They have lost their connection to their original data or idea. One central example is Aldrich’s Organizations Evolving. I think he is aware that he is drawing a very large picture, and I think he accepts this in the service of providing a framework. Nevertheless, from the original evolutionary perspective on organizations to construct an all encompassing theory like this one would, fairly, be said to have escaped its original phenomena. Aldrich, for example, moves from a phenomena – that organizations (like organisms) appear to evolve over time, to adjust with their environment and successful ones tend to be copied – to less grounded theory that organizations can be seen in an evolutionary context. Unresolved questions remain an the theory seems at times to be tenuous. For example, what does it mean for an organization to ‘reproduce” – what are its gene’s? Can one identify the “offspring” of an organization? The analogy has, I argue, slipped a little far from the observed phenomena, and the theory, therefore, in my opinion, is less valuable than it could be if more solidly connected to the phenomena.
The TCE literature, similarly, has moved to being a school of thought without very much phenomena or data to substantiate its argument. Williamson’s use of contract theory to deal with uncertainty in the future, for example, is far from Coase’s simple hypothesis and is arguably not grounded in phenomena but speculation (though it could be more grounded if propositions were drawn and a study done on this topic). Lastly, Fligstein’s book on Control, though it does make extensive use of data in, for example, the automotive industry, takes its hypothesis about government control and influence a bit far, in my opinion. While this is a valuable insight I do not follow him as far as he wishes to go. Hirshman’s book “Voice, Loyalty, and Exit,” as much as I enjoyed it, was a troubling example. On one hand it was pointing to specific things that would or did happen to organizations as they lost customers or declined. On the other hand it is hard to imagine how this could be tested or disproved. It is one of the pieces we read that was not strictly phenomena driven, perhaps more game theory driven, but escaped the troubles that this potentially holds for OT.
Have these “runaway trains” ‘gotten themselves into trouble?’ I think a theory which slips its phenomena too much risks being so broad that it becomes incompatible with other theories and ways of thinking. Thus, it either takes over and becomes a paradigm or it slips away. For example, TCE is arguable a contender in certain aspects of OT in being a major or dominant paradigm. But Fligstein’s argument, I think, would have benefited from a slightly less “explanatory” ambition. But please keep my relative ignorance in mind when I assert such sweeping opinions. Other theories, such as those of Selznick and Stinchcombe, have been picked up by other authors and used in situations which the authors never intended. But this appropriation even far from the phenomena, in these, cases, was appropriate. Stinchcombe in particular is an amazing example of a broad and insightful piece with enormous historical insight which is still the most cited piece in the organizational literature. His theory has become widely accepted, or at least parts have been, as a framework for thinking about organizations.
Karl Popper thought that one of the key attributes of a science was that it is testable and disprovable. I think this gets to the heart of some of Pfeffer’s disapproval of some theories that have slipped their phenomena. There is no longer anything to go back to and ask if the analysis was done correct, if it still holds 10 years from when it was hypothesizes, or if it is really true or merely sounds good. Organizations evolving, for example, would be a hard book to “disprove,” as would “markets and hierarchies” while the essays on economic sociology, or Selznick’s book on the TVA is at least amenable to a reexamination of the organizations or phenomena they spoke of. The use of testable propositions is one way that theories, even ones which do not directly use data, such as Meyer and Rowan or DiMaggio and Powell include propositions which can be tested by others and behind which they will stand. Selznick’s other book, Leadership in Administration, is another example of a wonderful book that one would be hard pressed to prove or disprove.
Pfeffer, as one of the early pioneers of the field, and one its greatest voices, is, I think, concerned with the power of the field and its progress. I think he believes that if the field does not stick relatively close to phenomena there will be less “building” less “consensus” and thus less of an OT “framework” which pushes the field forward. Economics, for example, has been greatly helped by the clarity of its framework. Further, I think, he sees the danger that if the insights of OT stick too closely to the phenomena, they will be stolen by other fields, such as strategy, who will broaden them and claim them as their own. Thus, I think he would not agree with the wording of part e) of the question, that OT theories ought to be “a set of self-contained and self-referencing propositions about selected aspects of organizations or a more open-ended and less organized set of ideas interpreting but not necessarily explaining organizational-level phenomena as they occur.” I agree that this is giving OT potential for useful explanatory success than it merits.
A good idea closely placed upon the phenomena will eventually begin to escape it as it is used more and more broadly. Furthermore, I think if this theory is not reestablished and constantly regrounded it can lose its way. But while this implies that Organizational Theorists ought to always remember that they must stay relatively close to the phenomena I also think that it is a field that has a great amount of power to speak in a predictive and generalizeable way about organizations. This struggle between theories becoming objects of discourse over time (especially as they slip out of their filed and are picked up by other less qualitative fields) is a natural one, and exists in many other fields as well. The struggle of any field is to reassert and fight to maintain an appropriate balance between truthfulness (closeness to phenomena) and usefulness (distance from phenomena/level of theory) – and not to fall into either extreme. It is not a steady state between these choices that a field seeks, but a constant dynamic tension in the field, a constant and precarious equilibrium. A field that maintains this fluxuating state will, like a democracy (which wavers between freedom and control), be constantly reminded of the value of their ideas because they are forced to defend and reexamine them. Thus, on hopes, they can capture much of the best of both choices.
Phin Upham has a PhD in Applied Economics from the Wharton School (University of Pennsylvania). Phin is a Term Member of the Council on Foreign Relations. He can be reached at phin@phinupham.com.