Topic

PowerPoint Does Rocket Science

-- Edward Tufte, September 6, 2005

-- Athel Cornish-Bowden (email), September 7, 2005

Both Nature and Science are widely available to serve as convenient models for technical reporting. I think Nature is a better model than Science.

The scientific journalism and the technical commentaries in Nature are also excellent models for more accessible technical reporting, such as that done at NASA.

-- Edward Tufte, September 7, 2005

Your assessment of the critical Boeing slide and the organisational/managerial decision-making context in which it was consumed is perceptive, detailed and deeply saddening. If only such opaque and misleading slides were the exception: in my experience they are far too common.

But I disagree with your conclusion. You suggest that a move to Microsoft Word and substantially text-based reporting will improve the situation, but a word of caution is required here. You rightly state that 'concisely written reports' should form the basis for decision-making discussions. But how many people are able to write the crisp, insightful, concise documents you envisage? I suggest that unless the writing skills of many thousands of professionals improves dramatically, high-level executives will continue to demand presentations. In my experience, the act of launching Microsoft Word ingnites in most professionals an urge to write lengthy, repetitive documents with peripheral appendices, duplicative introductions and misleading executive summaries, in which the key conclusions are at best difficult to discern and at worst so caveated as to be absent. Reading professional reports often requires ploughing through tens of badly-written (and often boring) pages in the (occasionally vain) hope of finding useful conclusions or messages. The authors of these documents generally overestimate by a substantial margin the time that the recipients have to read and understand them. Isn't it small wonder that time-starved decisionmakers demand slides?

Your targets should be woolly, imprecise thinking and misleading communication, no matter what the software package in which they appear. Word is no safer than PowerPoint from these twin evils. It seems to me that the real question is: how can we teach people to leave formulaic convention behind and to communicate crisply and accurately, no matter what the medium?

I'll end here, only hoping that you and your other readers will show mercy in commenting on the inevitable divergences between my contribution and the high standard of writing that I have described...

-- Will Judge (email), September 7, 2005

State your message in one sentence. That is your title. Write one paragraph justifying the message. That is your abstract. Circle each phrase in the abstract that needs clarification or more context. Write a paragraph or two for each such phrase. That is the body of your report. Identify each sentence in the body that needs clarification and write a paragraph or two in the appendix. Include your contact information for readers who require further detail.

-- William A. Wood (email), September 8, 2005

This was an entry at the National Review blog called The Corner at 1:22 p.m. September 8 [http://corner.nationalreview.com/].

If you want to see low levels of useful data per slide, let alone irrelevance to the task at hand [presumably saving lives threatened by Katrina], you could hardly beat those referred to below.

"On lunch at work, but still would prefer no identification if referenced. Thought you might like to experience what our elite fire and hazmat volunteers are going through. This is insane.

"It appears that all people under FEMA for over two weeks must take "awareness and prevention of sexual harassment," "equal rights officer orientation," and "valuing diversity" training programs. The programs total 3-4 hours.

"The mandatory training matrix is here: http://training.fema.gov/EMIWeb/downloads/Mandatory05Matrix.doc

"The letter adopting the matrix is here: http://training.fema.gov/EMIWeb/downloads/Mandatory%20TrainingII.doc

"Powerpoints for the trainings:

http://www.training.fema.gov/emiweb/downloads/DF506/DF%20506%20Sexual%20Harassment%20Visuals.PPT

http://www.training.fema.gov/emiweb/downloads/Df434/DF%20434%20Intro%20to%20Equal%20Rights%20Powerpoints.ppt

http://www.training.fema.gov/emiweb/downloads/DF416/DF%20416%20Diversity.ppt

-- John Liljegren (email), September 8, 2005

Continued Success, Ben-Zion Wasserman

-- Ben-Zion Wasserman (email), September 8, 2005

ET

-- Edward Tufte, September 10, 2005

I wasn't surprised to see my remark about Nature and Science contested but I was surprised to see who did the contesting, because what I see as the faults of these journals are exactly the sort of faults that you often criticize in your books. They both publish a lot of work that is fashionable and apparently exciting, but they don't insist on including the supporting information that allows readers to know exactly how the experiments were done -- half the time they wouldn't even allow authors to include this information because they would say it made the article too long. What happens in practice, therefore, is that high-profile authors will publish a claim-staking exercise in Nature or Science and then, if you are lucky, follow it up later in a journal of lower prestige with a "full paper" that includes the essential details omitted the first time round.

Let me quote (in suitably anonymized form) an e-mail that I received last week from a distinguished colleague in the US:

Thank you for bringing the (journal) paper by (authors) to my attention. I have not been keeping up on the literature (relevant comments below) and was not aware of it but absolutely agree with it. The paper by (other authors) (Nature, 2003) is pure bullshit, and the editors and reviewers responsible for letting it be published in Nature should hang their heads. (Name) and coworkers have hit the major problems (a totally incorrect assay procedure, highly suspect immunoblotting results) on the head in the third paragraph of his discussion....

This is just one isolated example, of course, but one out of many that one could cite.

-- Athel Cornish-Bowden (email), September 11, 2005

At some of these organizations, a technical report is called a "pitch" and is presented in 10-15 minutes, or presented simply as a PP deck to look over, or shown as a one-slide executive summary, or circulated by email-attached PP slides for the cognoscenti. Some of that reporting is done in a crisis; the Boeing PP slides were prepared in 2 or 3 days when the Columbia was in trouble but still flying.

Moving from the PP slide-format to the Nature-style concise report would be an enormous improvement for any applied technical organization. Fretting about the differences between Nature and the Proceedings of the National Academy of Sciences is not relevant to improving NASA workaday technical reporting.

Nature and Science publish about 15,000 authors and co-authors a year, which means, given their high rejection rate, they disappoint perhaps 50,000 aspiring contributors each year. That is a lot of enemies to make. Nature and Science rejections probably annoy and bruise more scientists than all other scientific journals combined. Very few scientific publications have high rejection rates, in large because publication is financed by page charges (not unlike a vanity press), paid for by the author's research grants or institution.

Just about everyone who has attempted to publish has their own personal collection of injustices to retail. These horror stories describe biased, incompetent, envious referees and idiotic editorial decisions--at least at every journal with a rejection rate greater than 0%. The anger and the whiny sense of entitlement occasionally exhibited by rejected authors can become rather intense (even experienced on this little board with a contribution acceptance rate of about 60%). Publication horror stories and associated gossip are rampant in the social sciences and humanities, where rejection rates for the top journals routinely exceed 90%.

Talking to journal editors, not just the multitude of rejected authors, will fill out this picture. Over the years I've served on a dozen editorial boards of research journals and have gotten a pretty good idea of selection processes. This wisest thing I heard was from the editor of journal with a 1 in 20 acceptance rate: it is easy to identify and reject the 90% of the submissions below the line; but for the top 10% [only half will be published], it's a lottery [depending on the quirks of the referees]. For NSF proposal reviewing (at least in the social sciences), it was usually very few star proposals which then led reviewers to ask how deep in the pool of routine dustbowl empiricism do we wish to dip? For journals and for grants, overall I was impressed with the care and integrity of the selection processes; I think most of us involved, other than the true believers, were seeking to find something, anything, that was good, novel, and true. I admire excellence in nearly whatever form it may take. For marginal submissions, those on the edge of accept/reject, non-meritorious factors may tend play a more important role in the decision, as it also does in faculty hiring in my experience.

The performance of a journal must be measured in aggregate and not merely by the embittered anecdotes of the rejected; that is why citations per published article and circulation numbers are relevant. Measured by citations per article, Nature and Science are close to the top, sometimes at the top. And they are by far the most widely circulated scientific journals. A measure of overall system performance is whether every minimally competent article gets published somewhere, if not in the most-cited and widely circulated journals. That is surely the case, since the median number of citations resulting from a published scientific article is zero.

[This thread should now resume its discussion of technical reporting.]

-- Edward Tufte, September 11, 2005

Athel's comments are spot-on. It is odd to see Dr. Tufte, of all people, invoke the popularity of Science and Nature as evidence in support of his notion that these journals provide excellent models for technical reporting.

Is it really enough that the correct result is obtained in the end, even if the audience is, like the audience watching a magician, left mystified by the steps taken to obtain that result? Is the "pitch culture" now acceptable if it is dressed in suitably sleek, technical- looking garb, rather than the strip-mall cheapness of a PowerPoint template?

There is no question that Science and Nature publish some spectacular science. They also publish some mediocre work, and some that might be reasonably described as craptacular. The problem here is that the extreme brevity of the Science and Nature formats in too many cases precludes even the expert reader from making a critical assessment of the claims presented, as Dr. Cornish-Bowden correctly points out (and no, I've never been described as "distinguished," i.e., that was not me).

As a consequence, in many cases the only way to critically evaluate a claim made in Science or Nature is to try to repeat the experiments oneself with the proper controls (not shown in the paper) included. This is also the case with journals that provide adequate space to describe methods, complete controls, etc., but in my opinion the problem's severity is substantially reduced. As a working scientist, I have to say that on balance, the technical reporting styles enforced by Science and Nature make it harder, not easier, to sort out what is being done in my field, and what it means.

-- Alex Merz (email), September 12, 2005