By now it’s clear to many readers that others have replicated my results. There’s even one at the blackboard.
Perhaps some industrious reader would consider it worthwhile to scour the net and find out just how often the results have now been confirmed. Then, of course, visit WUWT and ask Anthony Watts whether he’s willing to admit that the false claims in his document with Joe D’Aleo are wrong.
In the meantime, I’ll continue preparing for publication.
Open Mind 
But..but.. how could they possibly do it, if they didn’t have your code to look at?
I read your analysis with great interest.
I read the other blogs analysis also.
I find that I am better informed when I read both sides of the debate.
Keep up the interesting posts.
I was surprised to see how fast replication came.
I guess there are those who scream “post your code (so I don’t have to do the work myself) or you’re not doing science!” and those who simply got to work.
Oh, I meant to ask you after you initially posted on the new combination method. Have you considered testing it on synthetic data? I’ve done such a thing using different combination methods on synthetic data to see how well each method resists or succumbs to artificial inhomogeneities (urban warming, step changes, etc).
Anyone who believes there is no value in running someone else’s code using their data has clearly never worked in IT. It’s always (or should be) step1 for validation. Why? Because Tamino may say he ran code X for period Y to get graph Z, but it could be he was drunk at the time and graph Z actually came from code v.X-1.
Step 1 is doing the same thing to confirm the result. That is “replication.”
No one has replicated Tamino’s work – they’ve only approximated it based on his detailed posts. The differences may be negligible, and perhaps in climate science small differences are OK. If we were to SOX-audit this, you’d better match to the penny. Awfully difficult to do without the original code – unless the original code was exceedingly simple.
[Response: You’re might be right about what “replication” means in IT, but when it comes to science, not so.
My work has been replicated (in the scientific sense), and that constitutes much stronger confirmation than simply showing that I wasn’t drunk and using a different version than stated.]
Poor Anthony W. Now he will have “X” times as much work to do. He will have to demand acces to “X” times as much code, from “X” times as many sources, need to replicate “X” times as many results… So much extra work, in fact, that he may never be able to complete it… so sad, such a heavy burden…
Meanwhile, after re-inventing the first difference method or something like it, EM Smith is rambling on over there. I don’t have the patience to read his rambling, but I’m guessing he’ll eventually find some individual stations that were dropped and didn’t correlate well with the neighbors. This causes a sampling error in that region and a slight hiccup when using the reference station method, but we’ve already seen the big picture.
KenM // March 1, 2010 at 10:43 pm — I agree, but not for the reason you gave. Recently here it seemed advisable to get working code which had not been used for 15 years. The first step was to recompile the FORTRAN on the more modern computers and replicate the original computation using the original data. Ok, that all worked without changing a line of code. [Thank goodness, the stuff is almost unreadable, but fortunately the EE code writer is still around.]
Now this code is available for analysis of some newer data. But this (minor) effort is not for the reasons of replication, it is because a flood of newer data is anticipated to be coming soon.
KenM,
“Anyone who believes there is no value in running someone else’s code using their data has clearly never worked in IT”
I’ve work in IT for decades and it is clear to me that there is usually little value in running someone else’s code using their data. This is simply repetition – if everyone uses the same code it will have the same failure modes.
It is a far better confirmation of code quality if different implementations of the same algorithms get the same results. That’s why students of IT aren’t taught code by rote – they are taught algorithms. Only where different implementations get different results, or where the algorithms aren’t well described or there is some dispute about them, would it make sense to look at the code. Which is clearly not the case when we look at the ‘auditors’ efforts – they haven’t even attempted replication.
Also, clean room implementations – where people bend over backwards NOT to see other people’s implementations – are very common in IT.
Regardless, it is clearly a very powerful confirmation of scientific results when not only different implementations of the same algorithms, but also different analyses of the same or similar data, point to similar conclusions. Which is what we see here.
“I’ve work in IT for decades and it is clear to me that there is usually little value in running someone else’s code using their data. This is simply repetition – if everyone uses the same code it will have the same failure modes.”
That’s true. But that’s not the point. You are making the erroneous assumption that what Bob says he did, he actually did.
Bob says “I ran the widget revenue report for fiscal period six and got $1012.57. Joan in finance says she thinks that number is too low. Can you help?”
The very first thing you do is run that report for fiscal period six. When you come back and say “I got $2,129” Bob looks and says “Oh silly me – I must have put 2009 in as the year”
This is reality in IT land. I don’t doubt your credentials, but that’s what replication is and it is usually step one in troubleshooting. What other’s have done is validate Tamino’s result. If the question is broad enough, that’s good enough. (And by the way in this case what others have done is certainly good enough for me – I’m just nitpicking)
KenM,
OK, just playing devil’s advocate here. What if Bob and Joan got together and altered the program. If all you did was run the same program, you would simply replicate their error. Would it not be more thorough to go back to the original data, write a program to investigate the claim?
Replication is only step one. It gives you a reliable starting point for the rest of process.
As I said, I’m nitpicking words here and it’s really not of great consequence, but Tamino and everyone else is talking about is *validation*, not replication. FWIW I looked up “replication” and there’s a nice wiki article on what replication means in science (and in particular statistics). It’s no different than what I’ve been saying. You take and identical process and run it over and over again. According to the wiki:
“In engineering, science, and statistics, replication is the repetition of an experimental condition so that the variability associated with the phenomenon can be estimated. ”
In this case, we’ve got a static data set as input, so replication should produce exactly the same results. There is no variability associated with the “phenomena.”
The simple fact that no one has produced exactly the same results as Tamino is incontrovertible proof that his work has not been replicated – only approximated and to some extent validated. Personally I don’t think your work can be validated without first being replicated, but I suppose that’s debatable.
p.s. if Bob and Joan altered the program they would be fired :)
[Response: Utter nonsense. The wiki states “so that the variability associated with the phenomenon can be estimated.” Running the same code on the same data produces the exact same result, hence no variability at all — it’s worthless for the stated goal of estimating the variability associated with the phenomenon.]
KenM,
Since you said yourself that this would be running a deterministic program on a static dataset, I have to ask what value or confidence this would add?
Repetition to quantify variability is valid statistically and scientifically, but when there’s no variability, it’s kinda boring.
No value at all in that sense – I was using it to illustrate my point that Tamino’s work has not been replicated.
The Wiki definition:
In engineering, science, and statistics, replication is the repetition of an experimental condition so that the variability associated with the phenomenon can be estimated.
Note the title of this blog post.
Clearly I do not understand what ‘replication’ means, so I’ll just let it go. As I said before, it really is not that important at all :)
[Response: Replication is the repetition of an experimental condition. Not of the identical experiment, same subjects, same data, etc. And you conveniently ignored the “so that the variability … can be estimated” part. Does that tell you nothing?]
I did not ignore anything
[edit]
[Response: Let it go.]
Couldn’t help myself, I looked. Very bizarre place.
Well, so far Tamino’s optimal method, the CCC reference station method, and my simple average baseline method all produce comparable results, which is a good sign that the result is robust.
Carrot Eater had a good response to E.M. Smith over at WUWT the other day: Smith argued that these results don’t preclude divergence after 1992. Carrot points out that, since Smith had accused NCDC/GISS of “data manipulation and fraud” through dropping cooling stations, this would require them having invented a time machine as well as a temperature record :-p
Mind you, they did not initially talk about ‘cooling’ stations, which would at least make conceptual sense.
They were talking about cold stations; that notion is easily dismissed by simply noting that nobody averages together absolute temperatures. For the longest time, Smith was confused by the reference station method. Reading the relevant papers doesn’t seem to be part of his investigative method.
I’m guessing he’s moved past that point, but it’s hard to tell amidst the rambling.
Is there any chance someone much more skilled in blogging than I am could try to educate EM Smith on alternate ways to include those huge tables in posts as downloadable file attachments or something.
When I try to follow his explanations and get to one of those tables, I lose track of the thread by the time I’ve scrolled past it. Sigh, …
He’s using WordPress, there’s an “Add Media” link right there.
I don’t think “blogging” per se is his problem…
Sturat, easy quick copying is Ctrl-A and Ctrl-C key combo’s. In Firefox, copying tables is made easy by holding the Ctrl key, then mouse the columns/rows you like to copy.
Sadly, these aren’t HTML tables… these are like 1970 batch processing tables. I was thinking maybe it was a steampunk pride thing, but it appears to be more like cluelessness…
Not me. I want him to look every bit as stupid as he is.
Besides, he’s Watts and D’Aleo’s resident “computer expert”, so he doesn’t need any help from any of the rest of us, right? :)
“Anyone who believes there is no value in running someone else’s code using their data has clearly never worked in IT”
and
“If we were to SOX-audit this, you’d better match to the penny.”
I don’t think the difference is about works in IT, but the difference in approaches is about science world vs commercial world
If you do a mineral exploration study, you will probably collect the data once. Then you publish the results, and attempt to profit from whatever minerals you have found. A very direct motivation for fraud exists. I would assume that if you can inflate the amount of mineral by even a small percent, this is likely to translate into a similar small increase in the amount you can sell the mineral rights for.
So in such a case near enough isn’t good enough and you better be able to ‘replicate’ the answer to the nearest cent. Because any difference greater than one cent could be fraud that someone could be profiting from.
In science there is not the same motivation from fraud. No one is going to make some extra profit by tweaking the result of an experiment by a few percent. However an exact duplication of someone’s study may directly duplicate whatever mistakes they might have made. So replication by collecting your own data, doing your own analysis and finding a similar result becomes the golden standard.
Then someone who knows the rules of the commercial and mineral exploration world applies the same rules to the science world and cries fraud. Or more accurately points out that the climate science world is not following the same standards of the commercial world, and allows others to take these differences and cry fraud…
I think you’ve nailed one aspect of the disconnect that’s going on.
tamino’s work is always exemplary. With his
stuff and the stuff on skepticalscience I always
cream the denialists.
A quick question: am I right in thinking that
the results in the present effort could have
been anticipated by the work NOAA put
in to answer that idiotic surfacestations project ?
http://www.ncdc.noaa.gov/oa/about/response-v2.pdf
The issue of good/poor station siting is unrelated to what’s being studied here.
By the way, the NOAA guys wrote that up more formally as this paper, so you can read this instead:
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2/monthly/menne-etal2010.pdf
Chad:”I was surprised to see how fast replication came.”
Tamino obviously didn’t take my advice to stash the study in a secure place.
In the Hipparcos satellite data processing (stellar parallaxes) there were two completely independent teams with independent software bases doing each the full job. That’s what you do when it has to be right.
It’s not an either/or, Tammie. If you share your code, that does not prevent other people from more first principles replication (in the event of tree work, even resampling!). Often in problems, it is useful to look at things in different ways. Not only replication but also re-running.
and in the case of mathematical work (and I consider climate statistics mathematical), the exact program run shows the exact algorithm. It is COMMON to have vague or mistaken processes within methods descriptions (or to have mistakes in code, that are not in the methods). [edit]
[Response: I’ve got no problem sharing code, and when I publish this (which will be pretty soon) my code will be made available. But I object to those who claim that re-running my code is paramount — it isn’t, replicating the results with your own code is much better confirmation. And I object to those who want my code not because they suspect an error, but because they don’t trust me.
And frankly, it’s curious that there are so many demands for code from those who aren’t capable of writing it themselves, and may well not know what to do with it if they have it.]
TCO,
I do not see how running the same code on the same data tells us anything that reconstructing a similar algorithm and running it on a similar data set would also tell us. If the result is so dependent on the algorithm or dataset that it cannot be reconstructed in any other way, then I don’t think you want to put a lot of faith in it.
Personally, all I see in sharing code is a recipe for propagating errors from one research group to another.
I would also note that those who have screamed loudest and longest for the code did nothing with it once it became available.
I
Tammie: I agree that it is not PARAMOUNT. And that a lot of the yahoos are idiots and drama queens. Like I said, not an either/or, but an “and”.
Hey T. Speaking of data and publishing, are there any pointers you could give towards the literature regarding the below question that I came upon the other day…? “Can the reduction in the global warming trend over the last decade (1999-2009) be partially explained by an increase of energy used for phase changes in the melting of sea ice?”
[Response: I don’t know of anything about this in the literature. But I think I recall seeing a quick calculation that the energy required for ice melt is tiny compared to the energy imbalance due to greenhouse gases. But — I haven’t run the numbers myself.
The most fundamental error in the question is the statement that there’s been a “reduction in global warming trend over the last decade.” ‘Taint so.]
rayms,
The best estimates for ice melting come from the GRACE satellite–about 2 trillion tons of ice in 5 years. I did the math once. It amounts to a few percent of the radiation budget. Not entirely negligible, but probably not a major contributor.
Rayms: I also recall crunching the numbers once and I got a number even smaller than Ray’s. Less than 1%. However it is a complex question since some of teh energy that goes into melting the ice would have been lost if reflected back into space.
Dear Tamino,
Let me first express my gratitude for the debunking work you are doing. It should not be necessary, but it is. It is great that some people are willign to spend the time on this.
I have the following question (which may be dumb, but I am going to ask it anyway): I have the impression from your articles that temperature averaging is done over quasi-rectangular areas. Why shouldn’t one use a triangular grid (say constrcuted with Delaunay triangulation) ? The global average temperatur is then easily computed from the area of each triangles and the temperatures at the corners.
?
[Response: The only reason to use (quasi-)rectangular areas is convenience.]
Sorry if this is off topic but I just chose the latest post as a place to ask for some help. The popular press is in overdrive with “new” information about “Climategate” every day. Does anyone know if there is a website that follows this story, providing rebuttals to the latest “revelations”? Just looking for something to help with those workplace and family conversations that always start off by citing the latest headlines. Thanks!
[Response: This question belongs on the open thread; answers should go there.
RealClimate has run a series of posts on some of the nonsense allegations going around.]
Also check out Tim Lambert’s site. A lot of the “gate” stories come from one reporter in England who is a bit, um, sloppy–to say the least.
http://scienceblogs.com/deltoid/
An interesting article on getting sense out of a noisy signal. I don’t know enough to know if it is applicable to temperature data.
http://www.wired.com/magazine/2010/02/ff_algorithm/
jaydee says
“Making cents of noisy signals” is what climate science is all about, at least according to someone who should know, climate scientistMichaelTobis
[Off topic]
Eduardo Zorita (paleoclimatologist at the Institute for Coastal Research at the GKSS Research Center in Germany, running the Climate Onion blog) says: “I still think that Rahmstorf 2007 is wrong […] please comment with a statistician of your choice the Rahmstorf paper and the two comments to it that were published in Science. You may be surprised.” If I have understood him well, he questions he whole semi-empirical approach based on those two comments about the involved statistics.
You are the only expert in statistics that I “know”, Tamino. If you have looked at Rahmstorf 2007 (updated in 2009), I’d really appreciate your (brief) opinion about it. Thanks.
*References:
– Stefan Rahmstorf, 2007: A Semi-Empirical Approach to Projecting Future Sea-Level Rise. Science 19 January 2007: Vol. 315. no. 5810, pp. 368 – 370 | DOI: 10.1126/science.1135456
– Simon Holgate, Svetlana Jevrejeva, Philip Woodworth and Simon Brewer2 (2007): Comment on “A Semi-Empirical Approach to Projecting Future Sea-Level Rise”. Science 28 September 2007, Vol. 317. no. 5846, p. 1866 | DOI: 10.1126/science.1140942
– Torben Schmith, Søren Johansen and Peter Thejll (2007): Comment on “A Semi-Empirical Approach to Projecting Future Sea-Level Rise”. Science 28 September 2007, Vol. 317. no. 5846, p. 1866 | DOI: 10.1126/science.1143286
– Stefan Rahmstorf (2007): Response to Comments on “A Semi-Empirical Approach to Projecting Future Sea-Level Rise”. Science 28 September 2007, Vol. 317. no. 5846, p. 1866 | DOI: 10.1126/science.1141283
*Update of Rahmstorf 2007:
– Martin Vermeer and Stefan Rahmstorf (2009): Global sea level linked to global temperature. PNAS December 22, 2009 vol. 106 no. 51 21527-21532
[Response: I haven’t looked at the comments, and I’m kinda busy right now. Maybe later.]
George, ‘making cents’ is not the same thing as ‘getting sense’ or ‘making sense’
Thanks for your answer anyway, Tamino! ;-) I knew you were very busy with the GHCN analysis.
Cheers!
Just wanted to say that the denialsaurs are out in a Guardian comments section discussing the Parliament hearing yesterday. One cherry picked the past nine years showing cooling whilst claiming to be a physicist, and another says he does stats for the the financial sector, or some such, and after I pointed him here he started namecalling Tamino.
Guardian’s free to join. Anyone here fancy taking it to the general public?
http://www.guardian.co.uk/environment/2010/mar/02/institute-of-physics-emails-inquiry-submission?showallcomments=true#end-of-comments
Scroll up and look at physicist27’s and KMichaels’ comments.
E.g., “If Tamino is supposedly your guru statistician I would look around for a new one. Only a moron pretend statistician would assume that a linear regression chart has predictive abilities in it, especially one measuring TEMPERATURES for gosh sakes, as opposed to an entity that is plausibly sustainable, such as a proven rising trend in say fuel consumption.”
Any advice?
The poster who says a linear trend is inappropriate is an idiot! When looking at short time intervals, a linear trend is the only one that makes sense. You simply don’t have enough data to support a more complicated model AND to a first approximation (Taylor to be exact), any odd function will be linear. What is more, there is also a physical rationale for a linear trend–CO2 increases ~exponentially, while forcing increases logarithmically with CO2–result, approximately linear! The poster could not possibly be more wrong!
J Bowers write: “E.g., ‘Only a moron pretend statistician would assume that a linear regression chart has predictive abilities in it…'”
The analyses that Tamino does are for existing data to determine whether a trend exists. The “How Long” post addresses the “No warming since 1995” meme. See also Skeptical Science.
Future temperature projections are not based on extrapolating current trends, but on climactic models using scenarios such as increasing or steady CO2 emissions. Your correspondent does not seem to understand this, or does and argues otherwise.
Thanks to Ray Ladbury and Deech56 as well. Another Guardian reader posted your comments daring him to come here and continue, but the guy’s still sticking to his guns and namecalling, saying the method is a dead horse that’s been completely beat. Oh well. Thanks again.
J Bowers, I took a brief look at the comments, and they are just a pile of oft-debunked talking points; you can almost reply by just citing Skeptical Science numbers.
“Guru”, like “religion” is troll bait.
Actually ignore that last post. You all have better things to do.
J Bowers // March 2, 2010 at 10:39 pm — On the latest Open Thread I posted the decadal averages from the GISTEMP global temperature anomaly product. Looking at those, anyone can see a continued upward trend for the last 50 years. If you prefer such decadal averages in the form of a graph, NOAA has one and maybe GISS does as well.
I actually posted a WoodForTrees page I did from 2000, but it was ignored. Thanks for the reply. The Guardian comments can get pretty abusive, so time to watch a film I think ;)
I think the wiki article on reproducibility has a better description of replication (or reproduction) in the scientific sense:
Reproducibility is one of the main principles of the scientific method, and refers to the ability of a test or experiment to be accurately reproduced, or replicated, by someone else working independently.”
The results of an experiment performed by a particular researcher or group of researchers are generally evaluated by other independent researchers who repeat the same experiment themselves, based on the original experimental description. Then they see if their experiment gives similar results to those reported by the original group. The result values are said to be commensurate if they are obtained (in distinct experimental trials) according to the same reproducible experimental description and procedure.
Reproducibility is different from repeatability, which measures the success rate in successive experiments, possibly conducted by the same experimenters. Reproducibility relates to the agreement of test results with different operators, test apparatus, and laboratory locations. It is often reported as a standard deviation.
“If Tamino is supposedly your guru statistician I would look around for a new one. Only a moron pretend statistician would assume that a linear regression chart has predictive abilities in it, especially one measuring TEMPERATURES for gosh sakes, as opposed to an entity that is plausibly sustainable, such as a proven rising trend in say fuel consumption.”
Or in other words, since we know its impossible for temperature to have a gradual rising trend (unlike fuel consumption), then Tamino is a Moron for proving that tempratures are following a gradual rising trend.
Tragic Comedy.
On the subject of code, and Steve McIntyre:
SM yelled and screamed about the code for MBH 98/99. When Mann et al released the 2008 PNAS study, they also released the Matlab code.
So what happenned? SM criticized Mann’s code, and when attempting to replicate Mannet al, decided to roll his own for that reason. He ran into a bunch of problems, mostly due to poor understanding of the paper (and I suspect he didn’t go over Mann’s code very carefully, either, if at all).
And on top of all that, Mann’s code was better parameterized and structured than McIntyre’s.
Tamino (or anyone else who cares to comment): what do you make of this chronicle of the Mann vs. McIntyre story?
[edit]
[Response: I’ll give it all the attention it deserves.]
V.i.,
I second H.R. appreciation of your website.
If you are interested in me or my readers having a look at your McIntyre/Mann link, you could post it in a comment at a relevant thread at Deep Climate, e.g.
(Tamino, sorry for the digression).
V.i., nice website and good info you’ve got on the LeQuere work, glad to see it.
http://demon-hauntedworld.blogspot.com/
An indirect reflection on Inhofe’s issues with science above that post, too. Thanks for the link.
Thank you for your skill, time and passion devoted to this website.
Most of your analyses are fairly straightforward to me. I’ve spent over 25 years in ecological research. So phrases such as “principal components” or “time-series” or “autocorrelation” do not scare me a bit…I know immediately what I would need to do to start constructing a useful dataset and start asking pertinent questions. But people like me are not your audience.
Tamino, from what I have read you patiently plod through all the silly comments, and you apply appropriate statistical procedures, and you carefully explain why you apply those procedures.
In short you are a mentor in the very best sense of that word. You encourage people to teach themselves by evaluating data and by being skeptical themselves. Then you apply decision rules based on statistics.
Bravo. I think your planned publication based on analysis of freely-available data is about the best thing I’ve seen on the blogosphere to date.
Go with it, my friend (whom I’ve never met), go with it.
I’ve worked in software development for 45 years and I don’t recall ever seeing such a boneheaded misapplication of a concept and confusion over terminology as KenM’s. As it says right up top, what has been replicated are the *results*, which is done for very different reasons that *duplication* of software and data.
I’ve been writing and architecting commercial software for a number of decades, and I second this.
If you want to find out if the code does *what it should*, you don’t run it again on the same data and see if you get the same answer. Why do you think we have QA departments who *independently* construct their *own* tests from the specifications to see if it does what it should?
Furthermore, once we have QA tests, we don’t even *need* to run the code twice to see if we get the same answer – if we don’t, that fact will become obvious during QA.
The generalisation to methods described in published papers is left as an exercise for the reader.
And this is why the McIntyre-less efforts to audit the actual code (and even intermediate results) are (how shall we put this) deeply sub-optimal. Putting it another way, “auditing” is a *much* weaker error-detection method than independent replication.
Down, Boy! We can all make mistakes without being boneheads.
The role of replication in science is not blindingly obvious. Indeed, there are some sciences (e.g. ecology) where replication is not possible.
Hopefully KenM and others will take this as an opportunity to reflect on what it is that replication buys us in science and what the most efficient manner is to ensure this outcome.
P.S.
From KenM’s source, which he failed to cite, and apparently failed to read carefully:
http://en.wikipedia.org/wiki/Replication_(statistics)
“One finished and treated item might be measured repeatedly to obtain ten test results. ***Only one item was measured so there is no replication***. The repeated measurements help identify observational error.”
“Hopefully KenM and others will take this as an opportunity to reflect on what it is that replication buys us in science and what the most efficient manner is to ensure this outcome.”
I’ve had my own crack at this on my blog here. It seems to me that corroboration of results is far more important than verifying implementation details, especially when the existing corroborations imply those details don’t matter (like they do here).
The details that the ‘sceptics’ ask for in order to do repetition have nothing to do with getting the right answer or advancing science but everything to do with discrediting the existing research and by extension the researchers.
Frank,
An interesting post. It is astonishing to see the level of ignorance reflected in McI’s post:
“The ONLY way to strike at the team and the consensus is to demonstrate that the consensus scientists have arrived at their results incorrectly. You have to demonstrate flaws in THEIR work.”
ABSO-F***ING-LUTELY NOT! The ONLY way to make progress is to PUBLISH your result, whether it confirms or refutes previous work. Show the way forward rather than looking backwards! Science has operated this way for over 400 years! McI has a lot of catching up to do.
With regard to replication, I think that the most important role is reflected in the old saying:
A man with one watch always knows what time it is; a man with two is never sure.
An independent replication validates the overall results of previous work to the extent it produces the same overall results. However, to the extent that the method used in validation differs from previous work, it also serves to elucidate the extent to which the result depends on the particular method.
Mere reproduction only elucidates random errors. Independent replication also brings systematic errors to light.
Jesús Rosino et al.: For statisticians, educators, students and researchers: I have put one of the most comprehensive link lists for hundreds of thousands of statistical sources and indicators (economics, demographics, health etc.) on my blog: Statistics Reference List. And what I find most fascinating is how data can be visualised nowadays with the graphical computing power of modern PCs, as in many of the dozens of examples in these Data Visualisation References. If you miss anything that I might be able to find for you or if you yourself want to share a resource, please leave a comment.
Ray Ladbury – “Indeed, there are some sciences (e.g. ecology) where replication is not possible.”
Not “not possible”, just a lot harder. A lot of ecological principles are testable and replicable – that’s why ecology postgrads and undergrads spend so much time in the field (literally in fields…) counting stuff! There were experiments that have been running at my uni’s field stations in Berkshire and Cornwall for years (some nice plots inside rabbit-exclusion fences studying succession, some mussel studies that had annual data going back 40 years or more), and the results of these tied in nicely with the results of similar experiments conducted elsewhere studying similar principles. Similarly the woodland studies around Oxford Uni. And there are controlled lab-like ecological experimental facilities that have also been running for years, such as the Ecotron (where I spent part of a summer helping set up an experimental run, cutting sensor wires to standard lengths…). One doesn’t get the replication as quickly or easily in the field as you do with computer models but it’s wrong to say it’s not possible. Just harder and sometimes very tedious!
Tedious is exactly the word, since changing conditions change the survival (and reproductive rates of most organisms present (the best adapted have smallest variance?), the reproducibility in ecology is necessarily statistical (with quite high error limits compared to other sciences).
Charlie B.,
I certainly meant no disrespect to ecological science–or my wife, an environmental scientist, would kick my butt!
I should have been more judicious in my wording and probably said that repetition is impossible. I am actually very impressed that ecological science has found ways to control for variability even given continually changing conditions. It really shows the power and adaptibility of the scientific method.
jyyh – That’s true enough for many studies, yep, although mostly you’re looking at changes that are too short for NS to be an issue, or long enough that observed NS is part of the study! And a great chunk of ecology is population biology which is models and stats (in fact founded much of modern stats…), so plenty of stuff to do in the warm and out of the rain. :-)
Ray – Definitely agree with you that repetition is (usually) impossible! Glad to clear that up.
I’m no longer doing ecology, or in fact any bioscience other than for fun, but what I learnt 15 years ago in the UK is still largely applicable to understanding climate, especially here in Oz where both weather and climate are so in-yer-face…