Thanks for the extra info. You have some very clear cut nomothetic
intentions:
> The grids of experienced and less experienced faculty will be
> compared in terms of complexity and integration. Analyses will
> include extremity measures, variability of intensity, and interjudge
> agreement. Correlations will be calculated for constructs and
> elements in hgrtids of experienced and less experienced faculty.
> Factor analyses will be conducted as well.
However, I have grave reservations not just about whether you will
get a good response rate and valid data (a function of enthusiasm
against time!) I think you desperately need to simplify the
procedure and you need to give some very careful thought to how you
will parametrise "complexity" and "integration". Even extremity, and
interjudge agreement are not simple issues and I'm not sure what you
mean by "variability of intensity". Many of us, myself included,
believe that there is some superordinate construction of construing
along the line of "complexity" and that grids reflect it. However, I
am extremely unconvinced that it is easy to find well mannered
indices of this in grid data. You should have a look at the archive
of correspondence on this list around F. Reid Creech's study. See
the list archives at:
http://www.mailbase.ac.uk/lists-p-t/pcp/
for some of that fascinating correspondence. Reid and I continued
some of it off the list (like you, we worry about boring people). I
said to him:
-I still think you have bigger conceptual problems with parametrising
-complexity in grid data than just these issues that you are (rightly)
-raising. I gave up the issue in despair when I last had a bash at it
-as I came to the conclusion that complexity, even mathematically, is
-itself profound complex and multidimensional. (Our use of the word,
-"complex" not the modern, post-chaos use).
He was specifically concerned about the issue of finding constructs
that seemed highly negatively correlated and wanting to reflect one
of such pairs to get them handled more apparently appropriately in
his (cluster analytic) parametrisation of grid complexity. That was
the context for this from me:
=== inclusion starts here ====
That's fine and sensible but the problem is not about correlations
between constructs as just deciding to reflect one of two negatively
intercorrelated constructs won't solve your problem as the
(likely) multidimensional nature of your data mean that you can get
incongruent triangles of correlations, consider three constructs
A,B,C. These may intercorrelate:
A,B -ve
B,C +ve
A,C +ve
now you've got a problem cos if you flip either of A or B you are
creating a new negative correlation. I am quite sure that this
situation _WILL_ arise. It's a side effect of the scope in any
reasonable sized grid for multidimensionality and it is precisely
because of this that we have to impose clustering, MDS or PCA style
dimension crunching approaches. That's why I suggested you use the
crude approach of running a PCA then flipping the constructs with
negative loadings on the first PCA. That won't flip any other loadings on
the higher PCs (as they're independent -- actually, it may well flip
the whole of some higher PCs owing to the realities of finite
precision on computers but that will be a complete flip so of no
importance). What you're doing is artificially aligning all
constructs to point the same way on the largest possible dimension of
variation that can be extracted from the grid (under linearity
assumptions).
I've created a highly artificial example and analysed it
Here's the data:
1 1 1 1 0 -1 -1 -1 -1 -1,
1 1 1 1 0 1 1 1 1 1,
-1 -1 -1 1 1 1 -1 -1 -1 -1,
-1 1 -1 1 -1 1 -1 1 -1 1,
1 0 -1 0 1 0 -1 0 1 0
Here's the construct intercorrelation matrix:
CCORR C1 C2 C3 C4 C5
C1 1.00 -0.04 0.07 -0.11 0.02
C2 -0.04 1.00 -0.51 0.33 -0.43
C3 0.07 -0.51 1.00 0.22 0.22
C4 -0.11 0.33 0.22 1.00 -0.14
C5 0.02 -0.43 0.22 -0.14 1.00
Constructs 3, 4 and 5 illustrate what I'm talking about. No flipping
will remove those correlations (introducing a cut off other than at zero
correlation for flipping constructs makes these things less common
but they will still occur)
The components are:
EIGVAL PROPN_VAL
11.466677 PC 1 0.3464
9.6177054 PC 2 0.2906
7.7063676 PC 3 0.2328
3.9084995 PC 4 0.1181
0.4007509 PC 5 0.0121
The construct loadings are:
C_LOAD PC 1 PC 2 PC 3 PC 4 PC 5
C1 -0.7387 2.2952 -1.7480 -0.1751 0.0115
C2 0.0695 -0.4494 -0.5622 -0.0400 -0.6128
C3 1.6431 1.8378 1.2949 0.7952 -0.1167
C4 2.8644 -0.4227 -1.1365 -0.5641 0.0823
C5 -0.1067 0.7693 1.1689 -1.7105 -0.0682
This says to me that I should flip constructs 1 and 5 to get the
maximum possible alignment in on direction on the biggest possible
single dimension of variation in the data. I do this and get:
CCORR C1 C2 C3 C4 C5
C1 1.00 0.04 -0.07 0.11 0.02
C2 0.04 1.00 -0.51 0.33 0.43
C3 -0.07 -0.51 1.00 0.22 -0.22
C4 0.11 0.33 0.22 1.00 0.14
C5 0.02 0.43 -0.22 0.14 1.00
(problem, as we knew, doesn't go away)
C_LOAD PC 1 PC 2 PC 3 PC 4 PC 5
C1 0.7387 -2.2952 1.7480 0.1751 -0.0115
C2 0.0695 -0.4494 -0.5622 -0.0400 -0.6128
C3 1.6431 1.8378 1.2949 0.7952 -0.1167
C4 2.8644 -0.4227 -1.1365 -0.5641 0.0823
C5 0.1067 -0.7693 -1.1689 1.7105 0.0682
No signs or values of loadings on higher PCs changed at all as
predicted (other than the necessary flips on C1 & C5)
=== inclusion ends here ====
I think that raises enough of the very real problems about
parametrising complexity for now. I would strongly advise you to try
to put together some more specific nomothetic hypotheses about
"complex" and "less complex" patterns of inter-element similarity you
might see. I'd also strongly recommend you using fewer elements and
definitely fewer constructs and also recommend that you think very
seriously about imposing at least some constructs. Simple
volunteering of constructs without the triadic elicitation procedure
should be sufficient if you feel you need volunteered constructs.
Sorry to bore on (anyone who's got this far!)
Chris
Chris Evans, Senior Lecturer, ||| Psychotherapy Section
Cranmer Terrace ||| Dept.Ment.Health.Sci.
London SW17 0RE ||| St. George's Hosp.Med.Sch.
Britain ||| University of London
Tel/fax.: (+44|0) 181 725 2540 ||| Email: C.Evans@sghms.ac.uk
World Wide Web:http://www.sghms.ac.uk/mhs/psychotherapy/intro.htm
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%