What Color Is Your Cuneus?

Career counseling via voxel-based morphometry? With the U.S. unemployment rate at 9.5% as of June 2010, job seekers might be willing to try anything to gain an edge. As part of the Trends in Phrenology craze sweeping the field, the Johnson O’Connor Research Foundation appears to be capitalizing on the new cultural neurophilia:

The Johnson O'Connor Research Foundation is a nonprofit scientific research and educational organization with two primary commitments: to study human abilities and to provide people with a knowledge of their aptitudes that will help them in making decisions about school and work. Since 1922, hundreds of thousands of people have used our aptitude testing service to learn more about themselves and to derive more satisfaction from their lives.

Sounds noble, right? Although they are a nonprofit, JOCRF charges $675 ($750 in New York)¹ for a proprietary assessment battery. And a very preliminary morphometric analysis by Haier et al. (2010) features prominently on their homepage. The genesis of this structural MRI study is helpfully described on their website.

Relationships Between Aptitudes and Brain Areas

...In late 2006 at a professional research conference, David Ransom, exploring how our founder’s vision could be pursued by funding an outside researcher through the Johnson O’Connor Research Support Corporation, discussed with Dr. Richard Haier, a leading researcher on brain imaging and intelligence, the possibility of relating the volumes of defined brain areas measured with structural magnetic resonance imaging (sMRI) to performance on Johnson O’Connor aptitude tests. In the spring of 2007 Dr. Haier agreed to work on such a study, and in conjunction with Mt. Sinai Medical Center in New York, to conduct sMRI scans of 40 Foundation examinees, under the supervision of Dr. Cheuk Tang.

In the summer of 2007 a sample of Foundation clients aged 18 to 35 was recruited to participate in the study by having scans completed at Mt. Sinai. These examinees were selected in two ways. First, a solicitation letter was sent to former clients tested in New York in the previous year and a half. Second, new examinees were recruited in person when they came in for testing.

In January 2008 the goal of 40 examinees with completed sMRI scans and Johnson O’Connor test scores was met. Dr. Tang sent the brain-scan data for the examinees to Dr. Haier, and Chris Condon sent along the corresponding aptitude test data. Working with the sMRI scans, Dr. Haier used recently-developed technology called “voxel-based morphometry” to identify various brain areas and measure the volume of gray and white matter in each area.

In addition to the structural MRI, each participant received the following cognitive tests:

The eight tests in the JOCRF battery were: Inductive Speed (IS), Analytical Reasoning (AR), Number Series (NS), Number Facility (NF), Wiggly Block (WB), Paper Folding (PF), Verbal-associative Memory (VM), and Number Memory (NM). Each is described in Additional file 1: supplemental table S1 [.DOC]. These tests have been used in research on various aspects of cognition and intelligence [e.g., Schroeder & Salthouse, 2004].

A confirmatory factor analysis was performed using the entire psychometric database from 2002-2003, consisting of 6,889 people who had visited JOCRF for vocational guidance and aptitude tests. Combining this group with the 40 MRI subjects, the analysis revealed loadings for g (general intelligence) and four other factors: Speed of Reasoning (IS, AR), Numerical (NS, NF), Spatial (WB, PF), and Memory (VM, NM). Using standard voxel-based morphometry methods (Ashburner & Friston, 2000), the authors correlated gray matter volumes with each of these independent factors.

However, with n=40 the study was underpowered to produce much in the way of significant results, which accounts for why it was published in BMC Research Notes. I have no problem with that, once we're clear on the scope of the journal:

The aim of BMC Research Notes is to reduce the loss suffered by the research community when results remain unpublished because they do not form a sufficiently complete story to justify the publication of a full research article. A key objective of the journal is to ensure that associated data sets are published in standard, reusable formats whenever possible. Data sets published in the journal will be made searchable and easy to harvest for reuse.

Press releases and news stores were not very clear on this point, however:

Brain Scans Could Guide Career Choices

By Jeanna Bryner, LiveScience Managing Editor

Brain scans may guide a person toward the optimal career, new research suggests.

The results show people's cognitive strengths and weaknesses are linked to differences in the volume of gray matter in certain parts of the brain.

And this!

MRI challenges Myers-Briggs

By Rebekah Moan

Good news radiologists! There’s a new place to set up that MRI machine: the guidance counselor’s office. Researchers are starting to use MRI to document an individual’s ability to perform on vocational guidance tests.

I see.

There should be some sort of prominent disclaimer when the popular press reports on such preliminary findings, but we find nothing of the sort. Instead we'll have pushy overbearing parents clamoring for that MRI to give their kid the advantage needed to get into Harvard.

Let's return to the actual methods and results reported in the article (which is open access for all you science writers out there):

Given the limited statistical power of 40 subjects, we detail results at p<.001, uncorrected, in all the tables...; figures are shown consistently for all analyses at p<.01 uncorrected, to allow straightforward comparisons. Findings corrected using the False Discovery Rate (FDR) p<.05 are noted; no findings survived correction using Family Wise Error (FWE).

Table 1 gives the full list of brain areas that showed non-significant positive correlations between gray matter volume and the factors of g, Speed of Reasoning, Numerical, and Spatial. The Memory factor did show significant negative correlations with some regions, as shown below [note that a higher memory score was associated with smaller gray matter volumes]:


Table 1 (modified from Haier et al., 2010). Brain areas with significant negative gray matter correlations (p<.001 uncorrected) with the Memory factor. * BA is Brodmann Area, Talairach x, y, z co-ordinates; positive x values are in right hemisphere; Z is z-score; cluster size is number of voxels (blank entry denotes part of previous cluster); FDR is False Discovery Rate (blank entry denotes not significant p<.05).

How did the authors interpret these negative correlation?

The inverse direction of the gray matter correlations for the Memory factor was evident in both component tests, although we are unaware of any previous reports of inverse correlations between gray matter and other similar tests. ... Since there are previous reports of sex differences in the patterns of gray matter correlates to intelligence test scores, we recomputed these analyses for males and females separately. Only the males showed the inverse pattern. Why this should be the case is not clear. ... Since the sample sizes, however, were quite small for VBM stability (21 males, 19 females), we cannot interpret this finding with confidence. In general, VBM requires larger samples than 40 for stability, so this report is offered as an exploratory account of factor versus test correlates with gray matter in a sample uniquely characterized with a comprehensive test battery.

In brief: the authors are unable to interpret this finding! So don't rush out just yet for that MRI offered by the local entrepreneurial neuroradiologist who set up shop in the guidance counselor’s office.


Footnote

¹ Their website says this is their first fee increase since Jan 1, 2003. "For years now, we have been offering—and will continue to offer—the testing below our cost."

References

Ashburner J, Friston KJ. (2000). Voxel-based morphometry--the methods. Neuroimage 11:805-21.

Haier, R., Schroeder, D., Tang, C., Head, K., & Colom, R. (2010). Gray matter correlates of cognitive ability tests used for vocational guidance. BMC Research Notes, 3 (1) DOI: 10.1186/1756-0500-3-206

Schroeder DH, Salthouse TA. (2004). Age-related effects on cognition between 20 and 50 years of age. Personality and Individual Differences 36:393-404.

Odor Judge. From Ten Odd Jobs. Which One Do YOU Want?