Many psychology studies fail the replication test

No Comments

Scientific studies about how people act or think can rarely be replicated by outside experts, according to a new study that raises questions about the seriousness of psychology research.

A team of 270 scientists tried reproducing 100 psychology and social science studies that had been published in three top peer-reviewed US journals in 2008.

Just 39 per cent came out with same results as the initial reports, according an international team of researchers known as The Open Science Collaboration.

Their findings are reported in the journal Science .

The topics of studies reviewed ranged from people’s social lives and interactions with others to research involving perception, attention and memory.

No medical therapies were called into question as a result of the study, although a separate effort is underway to evaluate cancer biology studies.

“It’s important to note that this somewhat disappointing outcome does not speak directly to the validity or the falsity of the theories,” says Gilbert Chin, a psychologist and senior editor at the journal Science.

“What it does say is that we should be less confident about many of the original experimental results,” says Chin, who was not involved in the study.

Study co-author Brian Nosek from the University of Virginia says the research shows the need for scientists to continually question themselves.

“A scientific claim doesn’t become believable because of the status or authority of the person that generated it,” says Nosek.

“Credibility of the claim depends in part on the repeatability of its supporting evidence.”

Skewed picture

Problems can arise when scientists cherry-pick their data to include only what is deemed “significant,” or when study sizes are so small that false negatives or false positives arise.

Nosek says scientists are also under pressure to publish their research regularly and in top journals, and the process can lead to a skewed picture.

“Not everything we do gets published. Novel, positive and tidy results are more likely to survive peer review and this can lead to publication biases that leave out negative results and studies that do not fit the story that we have,” he says.

“If this occurs on a broad scale, then the published literature may become more beautiful than the reality.”

Some experts said the problem may be even worse that the current study suggests.

John Ioannidis, a biologist at Stanford University in Palo Alto, told Science magazine he suspects about 25 per cent of psychology papers would hold up under scrutiny, about the same “as what we see in many biomedical disciplines.”

Key caution

One study author who participated in the project as both a reviewer and reviewee was E J Masicampo, assistant professor at Wake Forest College in North Carolina.

He was part of a team that was able to replicate a study that found people who are faced with a confrontational task, like having to play a violent video game, prefer to listen to angry music and think about negative experiences beforehand.

But when outside researchers tried to replicate Masicampo’s study — which hypothesised that a sugary drink can help college students do better at making a complicated decision — they were not successful.

Masicampo chalks up the differences to geographical factors, stressing that the experiment showed how complicated it can be to do a high-quality replication of a study.

“As an original author whose work was being replicated, I felt like my research was being treated in the best way possible,” he says.

There are ways to fix the process so that findings are more likely to hold up under scrutiny, says Dorothy Bishop, professor of developmental neuropsychology at the University of Oxford.

“I see this study as illustrating that we have a problem, one that could be tackled,” says Bishop, who was not involved in research.

She urged mandatory registration of research methods beforehand to prevent scientists from picking only the most favourable data for analysis, as well as requiring adequate sample sizes and wider reporting of studies that show null result, or in other words, those that do not support the hypothesis initially put forward.

Scientists could also publish their methods and data in detail so that others could try to replicate their experiments more easily.

These are “simply ways of ensuring that we are doing science as well as we can,” says Bishop.


Journalism-by-robot may spread

1 Comment

Genes and musical abilities

No Comments

New research published today in the journal, Molecular Psychiatry, suggests our musical abilities may be determined by our DNA.

Are people getting dumber?

No Comments

Pressure to be intelligent is weaker for people today than it was for our hunting-and-gathering ancestors — and humans may be getting gradually dumber as a result, a scientist is propos­ing. [Read more here.]

Scrub jays found to react to their dead

No Comments

West­ern scrub jays sum­mon oth­ers to screech over the body of a dead jay, ac­cord­ing to new re­search from the Uni­vers­ity of Cal­i­for­nia, Da­vis. The ca­coph­o­nous bird “fu­ner­als” can re­portedly last for up to half an hour, though their pur­pose is un­known.

Funeral Held By Jay Birds

The Progressive Homogenisation Of Pop Musical Discourse

No Comments

A team led by artificial intelligence specialist Joan Serra at the Spanish National Research Council ran music from the last 50 years through complex algorithms and found that pop songs have become intrinsically louder and more bland in terms of the chords, melodies and types of sound used.

“We found evidence of a progressive homogenisation of the musical discourse,” says Serra. “In particular, we obtained numerical indicators that the diversity of transitions between note combinations – roughly speaking chords plus melodies – has consistently diminished in the last 50 years.”

They also found the so-called timbre palette has become poorer. The same note played at the same volume on, say, a piano and a guitar is said to have a different timbre, so the researchers found modern pop has a more limited variety of sounds.

Using evolutionary principles to create a pop song

No Comments

The perfect pop tune can be engineered by a computer program and refined with the input of listeners, according to a British study.

signs of human activity near lake albert, SA

No Comments

i’m just a creature of my semiosis

one lane, no cars

Notes on a uniform

No Comments

The anthropological bent has dogged me, one might say, for a goodly period of my life. Occasioned, no doubt, by most of my early and formative years being spent on the sub-continent where many ethnic identities and language groups made themselves apparent to each other by the laying on of identity signifiers, many of which took the form of clothing – as well as an array of related adornment of a less practical motivation. My later adolescence on the Antipodean continent-cum-island can also be considered formative, at least in this regard, in that the teenaged Sydney-sider, even in the far-off decades of the 60’s and 70’s, was regularly required to focus their attention on the outward signifiers of dress that identified the wearer as in alignment (or not) with the local power structure. This self-scrutiny and the accompanying scrutiny of others on the part of teenaged female high-school students at the time, was enhanced and given direction in 1970 by the first appearance in the media-scape of the highly colourful and yet not very bright DOLLY magazine. I well recall my first perusal of that initial edition, to the extent that I remember to where and with whom I was travelling, and by what means. My reaction at the time may have included scoffing, I may even have suggested throwing the publication from a window of the top deck of the double decker bus in which we were being transported away from rather than in the direction of our secondary school on a weekday.

At the end of the following year, for the school farewell ball, I brought, in lieu of a beau, my adoptive older brother, a person I had adopted to fill the space that a genetically-related brother might have occupied should I have had one – which indeed I should have had. The point of this short anecdote is that my adoptive brother, as my escort, had refused to follow the ruling set down by the school rule-makers, to wit, that escorts (interlopers, you must admire, into the all-female domain of our high school cohort) needed to be sporting a tie, worn in the appropriate fashion around the neck and collar. Instead, our rebellion was realised in a resistance of the local power structure through non-compliance with the dress-code, whereby he attended the event in a polo-necked jumper. We were rewarded for our efforts with a series of counter-resistant entry-level embarrassments in the form of discussions between my teachers and my escort. Since it was no longer the sixties at that juncture, I still wonder whether the polo was a good move to make.

In terms of formativeness too, I have not even mentioned the Mater’s influence on my later psychological make-up. Suffice to say that we (my sister and I) were subject to constant admonitions regarding the attire of exemplary others. And by ‘exemplary’, I do not necessarily wish the reader to imagine I refer to its regular positive connotation, but that attention was regularly drawn to those exemplars of style and taste which might advise us, in the words of those very clever mass media mavens Trinny and Susannah, what not to wear. With apologies for being less than precise here, we can summarise some of these instances of clothing error through the use of broader labels encompassing the main idea entailed. Certainly, for example, girls with fat legs should not wear mini-skirts. I personally could not agree more, and not primarily because I would hope to restrict anyone’s freedom to wear what they wanted – this would no doubt redound on myself in some way (I was born in India after all you see) – but because I am afflicted by a very nasty turn at the sight of visual arrangements which are not aesthetically-pleasing, which by the by has always been a great burden to both myself and to any companions, on occasions of traversing any locale where, for example, a McDonald’s has set up shop. Other combinations that one should avoid included that of dirty hair and a white collar, a stiletto and a bare leg, green and blue in the same outfit, a scarf tied about the rollers on the head, garish jewellery, and so on – these all administered by the Mater with a small disapproving grimace.

These notes that I offer here have been occasioned by a recent excursion stateside, where I attended a conference in NYC (a pretext, one might observe) in which context I was alerted once again to a phenomenon I am aware I have been subconsciously registering for some time, but have not systematically described as yet. My attention in this instance was arrested, or motivated perhaps, by the outward appearance of one of the presenters, whose self-satisfied but dull readings of the writings of some favoured performance artists while standing before blurry blown-up images of the same artists – all of course having lead intense and thwarted lives until their activities as performance artists meant that their subjugated and hitherto unappreciated inner selves had been released – caused me to interrogate in an extended fashion the basis for my sudden wave of displeasure at her delivery. On that score, I could uncover no satisfaction, but in the process I became aware of her vestimentary attributes, collocations of clothing items I have in the past remarked repeated in a variety of ways such that they can be considered variations on a theme, instantiations in fact of a genre, a conventional combination, an iconic reference to a potential state of identity rather than, say, indexical of an object.

/……to be continued

is that the war, or the war on?


series of tweets advertising a discussion, the exact meaning of which was unclear to me:

Tune in to Crikey Live at 12.30pm today. @BernardKeane, @BarrettBrownLOL & @mpesce will discuss the War on the Internet

i want to know whether that is the war ON the internet (in which case i’d be interested), or
the War [on the internet] (in which case i’d not).

so, maybe they should see if a Circumstance can be moved.
like, “On the internet, live at 12.30 pm today,@BernardKeane, @BarrettBrownLOL & @mpesce will discuss the War.”

but of course, even without grammar, we can use our brains, right?
i mean, which ‘war’ are we talking about here?
when someone uses presuming reference, then there must only be one item being referred to that we are all aware of.
so, as this is not the case wrt wars all over the place these days, it must not be one of those geographically-located wars with people getting killed and so on.

also, why say they’d be discussing anything on the internet? when it is clear where they’d be discussing it anyway. nobody says ‘on the internet’ in a tweet to remind people that the discussion will not take place, say, on the radio, or on t.v. or in a newspaper.

so, i’ve answered my question, and i should tune in.
except, i was not aware there was a war on the internet.
is there? who is waging it? why?
clearly, i am in need of information.

Older Entries

Get Adobe Flash player