The future of evaluation is within
When my friends and family ask me what my job is, I say something along the lines of, “I work at a public policy consulting firm. We mostly evaluate government policies and programs, and we gather our data mostly through surveys and interviews.” Knowing that I majored in biology, they tend to follow this by questioning whether I want to work in a field they see as so far removed from biology and science.
While the work I do to at ARTD feels meaningful and fulfilling, I’ve been left wondering… will I be able to access the parts of biology that I love while working in public policy?
Can we marry biology and evaluation?
When I saw that the 2018 ACSPRI Social Science Methodology Conference was discussing how to integrate social and biological research, I jumped at the opportunity to attend.
I attended the first day of talks at the University of Sydney on 12 December. The conference featured diverse expertise from a range of local and international backgrounds, including Naomi Priest from the Australian National University and Melissa Wake from the Murdoch Children’s Research Institute, Tarani Chandola from the University of Manchester and Michelle Kelly-Irving from the Université Paul Sabatier in France. Their talks highlighted ways biological concepts and methodologies have and can contribute to social research, with a focus on the use of biomarkers in social studies.
What are biomarkers?
To many, biomarkers are a new concept. Put simply, a biomarker is an objective measure of biological processes. For example, increased blood pressure can be used as a biomarker for increased levels of stress.
Chandola explained that the benefits of biomarkers include reducing measurement errors that can arise in surveys and being able to tell more holistic stories than those derived purely from self-reported data. In the example of stress, participants may underreport how stressed they feel or report not feeling stressed at all, while their blood pressure and other biomarkers suggest otherwise.
What do biomarkers mean for evaluation?
Because biomarkers can be used to determine the effects of the social environment on humans, they can be used in evaluation to provide enlightening data on complex social issues. For example, Kelly-Irving spoke about how adverse childhood experiences impact the physiology of the adult. According to her research, adverse childhood experiences tend to become more frequent with increasing social disadvantage. These experiences have neurodevelopmental impacts, which can affect health mediators (such as one’s likelihood of smoking, their BMI etc.) and social mediators (such as educational attainment), which all later influence mortality.
Priest also presented findings showing how different types of racial discrimination affect children’s health by increasing their BMI, waist circumference and blood pressure.
If biomarkers can be used as indicators of the effects of the social environment, and a public policy or program aims to affect social change, then biomarkers can be used to measure the effectiveness of said policy/ program. For example, if we were evaluating a program designed to combat adverse childhood experiences, we could compare the biomarkers of those who took part in the program with those who did not.
While it may seem complicated, biomarkers have the potential to enable social researchers and evaluators to draw clearer pathways between cause and effect. This is particularly useful in an increasingly complex social environment.
Are there any risks?
Despite their benefits, biomarkers don’t come without their own set of challenges. For one, sampling participants’ biology needs a heavy consideration of ethics regarding consent, risk, and data and sample security. Asking participants for samples of biomarkers may result in increased opt-outs and reduced sample sizes, therefore impacting the predictive power of significance testing.
Sampling methods for certain biomarkers may also be time and resource intensive, particularly when training is involved.
It’s also important that this sort of research isn’t used to exacerbate inequalities any further, i.e. that results are not presented in a way that provides certain groups with perceived biological evidence for their prejudices.
Using biomarkers also requires careful consideration of the conceptual framework. Before incorporating biomarkers into a project, evaluators must be certain they are valid indicators of the particular aspect of the social environment. Sometimes one biomarker is not descriptive enough. Indeed, Kelly-Irving found that a model comprised of many biomarkers was better than a model using just one in determining the impacts of adverse childhood effects.
What does the future hold for biology in evaluation?
The conference suggested that as our collective understanding of biological concepts, methodologies, and data increases, it will become easier to integrate the biological and the social. Social research and evaluation will benefit from a rich data source, with potential to support our understanding.
Generation Victoria, an ambitious project directed by speaker Melissa Wake, aims to gather biological and social data from all children born in Victoria between 2021 and 2022 with the goal of addressing social epidemics like school failure, obesity and mental health. Far-reaching projects like this become possible by merging the biological and the social.
On a more personal level, I find it exhilarating that I might be able to marry my passion of biology with the work I do at ARTD. Moving forward, I aim to look for and shape opportunities to integrate this thinking into our work, and I encourage you to do the same.
In using our own biology as measures of the effects of the social environment, the future of evaluation is, quite literally, within us all!
