eve_prime | This is priceless! (Reply)

This is from a critique of social psychology (and, in passing, economics), but I think anybody I know would have fun reading it.

The Martian Institute or Foundation for Furthering Science (MIFFS) Earth Sport Section (MIFFSESS)was convening for its 10th year. Up to this time, the Research Program had been entirely devoted to a thorough study of one simple earth sport that the earthlings call tennis. Progress was rapid, and many laboratories were engaged in the enterprise. Thus, it was quite a shock when a few scientists at the 9th MIFFSESS meeting suggested that MIFFSESS support research on the uninvestigated sport of football, at some cost to the tennis program. The tennis researchers pointed out, with some justice, that they had made great progress and now understood the scoring, physics, and other aspects of the sport. Yet there were still many problems to be tackled in the microanalysis of the game. There was, for example, the well-known “yellow ball problem.” A yellow ball was used on only some occasions, and no one could predict this distinct occurrence. Pigment analyses of the yellow ball were just beginning. “Why,” asked the tennis workers, “commit money to the murky enterprise of football when such good problems remain with tennis?” Nonetheless, in Year 9 a small amount of money was budgeted for the following year for the investigation of football. And now, the Committee had to evaluate the proposals.

The Committee was faced immediately with some fundamental disagreements among the applicants. Some claimed that the essential elements of the sport were six creatures with black and white striped costumes. They were the only participants who appeared to be on the field at all times. Others focused on the more than 50 creatures, some sitting, some running, each with his own number. Some claimed that the ellipsoidal object noted in some observations should be the focus of study, but others pointed out that this object was rarely visible and probably didn’t matter. In the end, the Committee agreed that the numbered creatures might be the best bet for study; because they had numbers, they could form the basis of precise quantification. For unnumbered participants, one might have to make up arbitrary numbers.

One proposal suggested correlating two measurable variables: the number of the players, an incontrovertible datum, and the percent fat, of known biological importance. Other proposals suggested electrical rather than biochemical analyses. One group proposed use of the standard electroencephalogram (EEG) technique. Each player would be wired up, and the total set of generated potentials for all of the players would be measured with a computer. The investigators worried about tripping on wires, but came up with the clever idea of using a blimp over the stadium, from which all the wires could be suspended. Yet more clever proposals did away with the individual EEG and proposed a macro approach, a total integrated reading, taken from the ellipsoidal extremes of the stadium itself. Another set of experimenters proposed to set up an animal model of football.

A group of economists proposed a model for football on the assumptions that (a) each player was totally independent of any other, (b) all actions in the game were symmetrical, (c) there was no change over time in the activities of any player or team, (d) all players were seeking the same goods, and (e) all players operated under the same constraints.

One group encouraged the search for invariances, and impressed the committee with the preliminary finding that the summed numbers of the players remained roughly constant through the game, although players came in and out. In the search for order, they pointed out that one should begin with what was apparently the most structured aspect of the game: the grouping of players in a circle, in fixed order, every minute or so. This was followed by another ordered formation, and then by an apparently disordered set of movements, probably the players “letting off steam.” Plots of position in the circle against player’s number seemed reliable, and a good point of departure. It was proposed that these observations be followed by detailed analysis of foot and hand positions of the players, in the circle and after, to build up the elements of the game.

There was one proposal that was easy to reject. It stood out as the one that failed to follow the basic scientific requirement of objectivity and quantification and was not model driven. The authors (from the fringe of science, at best) proposed to simply observe the general flow of the game and to supplement and guide these observations with interviews of the players in an attempt to find out what the game was about. They proposed to ask players such open-ended questions as: “What is the purpose of the game?” “Is the ball important?” “Why do the players move toward one end of the field for a while, and then to the other?” The Committee unanimously rejected this proposal, supported in this decision by unanimously negative reviews from tennis researchers. The grounds for rejection, contained in what might be called the quintessential pink slip, were many:

1. The study relied in large part on verbal reports, which were of questionable scientific status. Why, for example, should one believe a player’s claim that he moved to the right to misdirect other players or that the rarely visible ball was the center of activity?

2. Worse, the reports were retrospective. Players were not asked while they were playing, but after a game, reducing even further the reliability of the results.

3. There was no control. At a minimum, it would be necessary to question a group of control people who were not familiar with football.

4. The authors were unaware of the importance of social desirability. To be sure that the informants were not making up stories, the Crowne–Marlowe test should be administered.

5. At best, the research proposed involved only a single study.

6. The study might not produce interpretable data.

7. The investigators had no model for football; they proposed simply to explore it. It was well known that observation not linked to model or theory would be useless and unable to discriminate essential processes from trivial events.

8. The authors did not make clear what were the dependent and independent variables.

9. The authors failed to rely on, or even utilize, the only reliable route to understanding: experiment.

10. The authors failed to describe how they would statistically analyze their data (if they could get any). In particular, it was not obvious how the authors could perform an ANOVA on their data. One reviewer suggested that they could use football and tennis as the categorical independent variable, but even this helpful reviewer couldn’t think of a dependent variable.

11. One reviewer thought of a clever alternative account of any data the authors might gather. The reviewer noted evidence for a theatre tradition on Earth, in which what were essentially imitations of real life were portrayed. Perhaps, the reviewer proposed, all the authors would be describing was such a theatrical portrayal, with considerable distortions, no doubt, of the actual reality.

If the authors were “successful” on their own terms, they would have accomplished merely a pushing off of the fundamental problems. To say the purpose of the game and the moves of the players were represented mentally in the heads of the players was not progress, because we would then have to study the representation in the heads.

There was another proposal asking for funds to explore books in libraries on Earth in the hope that some information on the game would be unearthed. Because work on Earth libraries was in its early stages, the proposers would need a year or more of support to try to find the material. The proposal was rejected. Although most Committee members agreed that it might uncover valuable information, it could not be funded because it did not involve the discovery of new facts through research. After all, the proposed findings were already in books, somewhere.

And so it was that a decade of studies of the arrangement of players in the football huddle was begun, along with an analysis of the biochemical and electrical events underlying this circular event.