An open critique of the Nieuwoudt et al (2019) study on coloured women

By the Division for Research & Methodology (DRM)

The Psychological Society of South Africa’s (PsySSA) Division for Research and Methodology (DRM) is committed to the teaching, practice and promotion of relevant, appropriate and ethical scientific research in South Africa. We are disturbed by and strongly opposed to the practice of misusing racial classification in scientific research and the consequent perpetuation of stigma, discrimination and racism within our society, as exemplified in the recent publication of, Age- and education-related effects on cognitive functioning in Colored South African women, in the journal, Aging, Neuropsychology, and Cognition. The article by Nieuwoudt, Dickie, Coetsee, Engelbrecht, and Terblanche (2019) draws on colonial stereotypes of ‘Coloured’ women, and portrays them as intellectually deficient, making broad, reckless and injurious generalisations on the basis of a flawed methodology. The authors claim that “young to middle-aged Colored women present with low cognitive function and which is significantly influenced by education” (p. 1).

The statutory regulatory body, for all health care professions, such as medicine, psychology and its speciality neuropsychology, the Health Professions Council of South Africa (HPCSA), has standing ethical policies for health researchers (HPCSA, 2008). These policies are aligned with the Constitution of South Africa (Constitution of the Republic of South Africa, 1996a) and the Bill of Rights (Constitution of the Republic of South Africa, 1996b). As researchers, we must be held accountable to these ethical guidelines in theory and in practice. The first and foremost prerogative of research is the protection of participants through the core ethical values of non-maleficence, beneficence, autonomy, confidentiality, and justice. Our South African Bill of Rights provisions. include dignity and other universal precepts, that have become a central part of our polity in the twenty five years of our democracy. We do not believe that the aforementioned study has “placed the best interest of the participants above all other interests” (p. 7) as set out in the HPCSA General Ethical Guidelines for Health Researchers (2008), nor have they recognised that the researchers are “in a position of power over participants and should avoid abusing their position” (p. 7). Above all, the dignity and other human rights of participants as well as a critical part of our South African society have been violated through such questionable research masquerading as science.

The uncritical use of race in research

The authors have unjustifiably and exploitatively used the apartheid-inspired understanding of race which divided South Africans into so-called four hierarchically-arranged population groups, where those identified as ‘Whites’ were considered the top population group, while ‘Coloureds’ and ‘Blacks’ the bottom groups. Employing the apartheid understanding of race groups, the authors conveniently selected a group of ‘Coloured’ female participants, conducted their research and proceeded to generalise to the greater ‘Coloured’ female population.

The starting premise of ‘Coloured’ women being an understudied population, without the critical engagement of race and ethnicity, is an insufficient rationale for selecting participants based on their race or ethnicity. The use of apartheid-inspired (and thus politically motivated) constructs of race to select participants, conduct research and generalise to the greater ‘Coloured’ female population is baseless, largely damaging and does a disservice to scientific inquiry. By doing so, the study portrays all women racially classified as ‘Coloured’ as a homogeneous group inevitably linked to a particular social class. This is, in fact, blatantly argued by the authors in the introduction with the claim that “the Colored community is, in terms of social class, considered the most homogenous group in South Africa and are generally described as a poor, lower working-class community” (p. 1). The authors state that the post-apartheid government has ‘attempted’ to resolve inequalities of the past, but we must ask ourselves how this study has contributed to the resolve of inequalities of the past?

We recognise that race categories are still used in state and parastatal reports, the most prominent being the census. The objective of such use is meant to be the redress of historical racial inequalities. It is also necessary to recognise that the post-apartheid government has generally shown a lack of imagination when it comes to thinking beyond racial categories and slavishly follows apartheid designation, continuing to understand South Africa as made of four races. At the same time, cognisance should be taken that ideologies and constructions of race can be a legitimate topic of social scientific and philosophic research. However, we have to caution those who are considering the use of race (or ethnicity, which the authors also use) as biological markers or characteristics to seriously consider the implications of such use, especially as these may impinge on human rights and the quest for a truly free South Africa unfettered from its distasteful apartheid past. The discriminatory power to stigmatise women in South Africa by the colour of their skin suggests that the authors have not critically considered the implications of their interpretations. We oppose the perpetuation of harmful and discriminatory stereotypes and we stand against the use of scientific research to promote these stereotypes, whether consciously or unconsciously.

Fatally flawed study methodology

The study methodology is fatally flawed, in terms of sampling, instruments, data analysis, and the interpretation of the data. Each of these are discussed below.

Sampling

Although the authors declare, at the end of their paper, that the study is limited by the small sample size of 60 women, sampling procedure, and inability to generalise to the larger ‘Coloured’ population in South Africa, they also go on to say that the study provides “novel and valuable insight into the cognitive performance of Colored women” (p. 14). This claim is unbelievable. There is neither insight, value nor anything new about making racist generalisations about Coloured women. It is important too, to note that this is the first, and only time, the study claims its limited generalisability. Furthermore, when the sample was divided into four arbitrary age categories, the number of participants within each group further decreased, with the largest number of participants falling within the 50–64 age category (n = 16). Following this trend, the authors split the sample into two education groups, leading to an unequal distribution across the conditions, with 41 participants in the low education group and 19 in the high education group, making the comparability of groups questionable. Methodologically, the practice of making broad generalisations to a population requires a sample size estimation with considerations for population size, confidence intervals and standard errors as well as sampling procedures that use probability sampling and ensure geographic representation. Of course, even if the researcher had followed appropriate sampling procedures, the flagrant disregard for ethically and morally sound research would ensure that a study of this nature should not have been undertaken anyway.

Instruments

One of the instruments drawn on by this study, the Montreal Cognitive Assessment (MoCA) (Nasreddine et al., 2005), has been shown to yield results that are fundamentally flawed in the South African context. Robbins et al. (2013) found that HIV-infected Xhosa-speaking South Africans tested with this instrument performed similarly on several tasks to North Americans aged 70 and over who had Alzheimer’s disease and so concluded that the normative data is inappropriate for use with a South African sample. The MoCA takes approximately 10 minutes to complete and is considered a brief screening tool. Coen, Robertson, Kenny, and King-Kallimanis (2015), in an Irish sample, showed the inapplicability of the tool outside of the American context when used without validation procedures, and found that “the MoCA should not be viewed as a substitute for more in-depth neuropsychological assessment when domain-specific information is required” (p. 18). A more recent study by Beath, Asmal, van den Heuvel, and Seedat (2018) attempted to validate the MoCA in South Africa, and found the instrument to be “fairly reliable” but concluded, “certain domains and items are needed to improve the differentiation between normal ageing and MCI. Until such time that a culturally adapted version of the MoCA has been developed and validated for this population, we suggest lowering the cut-off score to 24 in order to reduce false-positive diagnoses” (p. 1). All three studies suggested that the instrument be used in longitudinal research. The study by Nieuwoudt et al. (2019) was cross-sectional and used cut-off scores from the normative data of adult Americans, which directly contradicts recommendations based on existing scientific evidence for the South African context.

Data Analysis and Interpretation

The authors used multiple statistical tests with post hoc analyses to draw conclusions. However, due to the contextually invalid nature and inappropriateness of the MoCA instrument, the conclusions drawn from the data cannot be treated as scientific evidence. Additionally, the authors include the results of cognitive scores in relation to American normative data, which is non-sensical given the scientific evidence that shows the heterogeneity across and within populations and the need to adapt the MoCA. In the discussion, the authors attribute low cognitive functioning to a “combination of low education level, poor quality of education and socio-demographic factors such as ethnicity, employment, marital status, income and health status” (p. 10). The authors furthermore support their ‘findings’ with a source stating that, “Peltzer and Phaswana-Mafuya (2012) reported that Colored and Black African older adults achieved worse cognitive scores than White and Indian/Asian older adults” (p. 10). This link to supporting evidence regarding racial attributions is weak and is made only further questionable by their lack of such evidence in their data and the inability of the study’s methods of analyses, namely, t-tests and ANOVAs, to allow for attributions between variables.

Call for all stakeholders to promote ethical research and practice

Given this study’s problematic rationale, methodology and its reproduction of harmful colonial stereotypes of African women, we believe that this study contributes to the perpetuation of discriminatory and racial stereotypes. We question how this flawed research has contributed to scientific knowledge and whether a credible peer review process was followed leading to its questionable publication in a journal that ought to know better.

We ask that Stellenbosch University explore the shortcomings in research management, particularly ethical clearance and student research supervision, which enabled research of this nature to progress from implementation to publication. In addition, it is insufficient for an institution of higher learning to simply foreground the principles of academic freedom and peer review within the scientific community, as the mechanisms that act as checks and balances around research content and quality. It is indeed incumbent for such institutions to exercise appropriate oversight and sanction in instances where the principle of academic freedom violates the rights of others, or where there is clear peer review that contests the validity of the research. The National Research Foundation, as the funding body, should also review its systems to understand how a project of this nature was funded by the very body set up to transform research in South Africa. The specific journal in question, Aging, Neuropsychology, and Cognition, should urgently interrogate its own peer review processes to ascertain how such an article could have passed as a credible scientific research paper.

We call for the article Nieuwoudt, Dickie, Coetsee, Engelbrecht, and Terblanche (2019) to be retracted, by the journal Aging, Neuropsychology, and Cognition.

Furthermore, we hereby call upon:

researchers and scientists to reflect on their own personal biases and conceptualisations of race with the aim of self-development, to reduce and eradicate racism and discrimination, which can be manifested in their research teaching and practice.
teachers and supervisors to be conscious in their knowledge exchange with students and to promote the production and use of relevant and ethical science.
institutions of higher learning to take responsibility for the management of knowledge produced by staff, students and groups within the organisations and contribute to their training in research integrity, social relevance of research and good ethical practice.
ethics committees to review proposed research studies with a critical race theory lens and to implement policies and procedures with the aim to remove potentially discriminatory research based on gender, race and ethnicity.
grant agencies to be sensitive to issues of race and discrimination in research and communities of study, and to request detailed information from applicants regarding the rationale for using demographic markers such as race as a variable in research.
journal reviewers and editors to be more vigilant in the reproduction and dissemination of research that may strip research participants of their dignity and perpetuate discrimination within society.
for all knowledge consumers to familiarise themselves with the content of articles and research findings and fact check information before sharing and distributing to others, as well as consider the implications of sharing.
the department of higher education to not reward such research which does not contribute to scientific knowledge or the good of the public.

We are a nation of diversity and we treasure our positive differences and must nourish them. We have the power to use our unique individual strengths and expertise to uproot historical seeds of racist and sexist discrimination, and stereotypes, and together build a just and fair society. We pledge to continue our support for ethical research and practice in South Africa and encourage the public, academics, educational institutions, funding organisations, the Departments of Science and Technology and Higher Education, community organisations and other professional societies to stand with us.

This statement was written by the The Division for Research and Methodology, for the Psychological Society of South Africa, and endorsed by the Presidency of PsySSA. For reprints contact [email protected]. Follow the DRM on Twitter @DRM_PsySSA

References

Beath, N., Asmal, L., van den Heuvel, L., & Seedat, S. (2018). Validation of the Montreal Cognitive Assessment against the RBANS in a healthy South African cohort. South African Journal of Psychiatry, 24(1), 1–1. DOI: https://doi.org/10.4102/sajpsychiatry.v24i0.1304
Coen, R. F., Robertson, D. A., Kenny, R. A., & King-Kallimanis, B. L. (2016). Strengths and limitations of the MoCA for assessing cognitive functioning: Findings from a large representative sample of Irish older adults. Journal of Geriatric Psychiatry and Neurology, 29(1), 18–24. DOI: 10.1177/0891988715598236
Constitution of the Republic of South Africa. (1996a). Constitution of the Republic of South Africa. (Pretoria, Government Printer). Retrieved April 22, 2019, from: http://www.justice.gov.za/legislation/constitution/saconstitution-web-eng.pdf
Constitution of the Republic of South Africa. (1996b). The Bill of Rights. (Pretoria, Government Printer). Retrieved April 22, 2019, from: https://www.gov.za/documents/constitution/chapter-2-bill-rights
Health Professions Council of South Africa. (2008). Guidelines for good practice in the health care professions: General ethical guidelines for health researchers. Booklet 6. Retrieved April 22, 2019, from http://www.hpcsa.co.za
Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., & Chertkow, H. (2005). The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53(4), 695–699. DOI: 10.1111/j.1532-5415.2005.53221.x
Nieuwoudt, S., Dickie, K. E., Coetsee, C., Engelbrecht, L., & Terblanche, E. (2019). Age-and education-related effects on cognitive functioning in Colored South African women. Aging, Neuropsychology, and Cognition, 1–17. DOI:10.1080/13825585.2019.1598538
Robbins, R. N., Joska, J. A., Thomas, K. G., Stein, D. J., Linda, T., Mellins, C. A., & Remien, R. H. (2013). Exploring the utility of the Montreal Cognitive Assessment to detect HIV-associated neurocognitive disorder: the challenge and need for culturally valid screening tests in South Africa. The Clinical Neuropsychologist, 27(3), 437–454. DOI:10.1080/13854046.2012.759627