A citizen science approach to the assessment of pragmatic language in adolescents

This pilot project will attempt to provide a proof of concept for a Citizen Science approach to better inform the assessment of pragmatic language in adolescents.  Pragmatic language is the ability to use and understand implicit meaning during social interaction. A number of standard tools exist to assess pragmatic language, but speech & language therapists find these tools inadequate for that age group, which seems to be characterised by very different ‘norms’ to other age groups. We will facilitate the creation of videos by young people from Bradford, on their experience of pragmatic language. This will inform a critical review of existing assessment tools, and lay the foundation for a larger-scale project. 

The pilot is funded by the University of Leeds' Cultural Institute. It is a collaboration between the University of Leeds, the University of Bradford, and political theatre company Common Wealth (Bradford and Cardiff).

Key findings from Q-BEx project

We have created a customisable online tool which researchers, speech & language therapists and teachers can use to better understand the language experiences and language background of bi/trilingual children, and to inform the professional evaluation of their language support needs. The tool’s design was informed by an international, cross-sector Delphi consensus survey (including representatives from 29 countries), the comprehensive review of existing tools, best practice identified in the psychometric literature, and consultation with bilingualism experts (research and practice). The tool consists of an online questionnaire and back-end calculator. The questionnaire can be customised according to professional users’ needs in terms of level of detail, type of respondent and language of administration, and is available in 23 languages, with 5 more to be added shortly (far exceeding our original target of 13).    The tool was validated using newly collected data from 299 children from 3 countries (France, the Netherlands, and UK), between the ages of 5 and 9. This includes data from the full questionnaire, as well as direct measures of proficiency in the societal language (French, Dutch, or English) and of relevant cognitive skills.     Exploiting advanced quantitative methods, we identified complex associations among the Individual Difference variables provided by the Q-BEx questionnaire, when considered as predictors of language proficiency. These associations indicate different types of profiles practitioners might encounter when assessing multilingual children.       We demonstrated the practical benefit of composite indices of Richness of Experience as predictors of language proficiency, and demonstrated how these indices share common and specific information among multilingual children’s societal vs home languages.     Using an information-theoretic approach, we identified the optimal level of questionnaire detail required to predict language outcomes in multilingual children (as represented in our validation sample).  This evidence will guide professional users to choose the level of detail to implement in the questionnaire, given their needs and constraints.     Risk for language impairment is incorporated into Q-BEx via a few simple questions about early language development and current oral skills in each language.  A Concern Score is derived to identify children likely to be at risk for language impairment. By recruiting some of the children in the validation sample from speech and language therapy clinics, we were able to demonstrate the usefulness of this Concern Score by how well it flagged independently-known-to-be-at-risk children, and by how well the Concern Score predicted language performance in the societal language in all the children flagged by that score.      The Concern Score is incorporated in the individual child reports automatically generated by the Q-BEx platform. These reports also include information about the child’s amount of experience in each language (current and cumulative), parental estimates of the child’s proficiency in each language, and indices of Richness of the child’s experience of each language. Evidence-informed guidance is provided for the interpretation of the reports.     The validation of the questionnaire also included a qualitative assessment, analysing the validation study data loss, inconsistencies in data, and identifying unlikely scenarios reported by respondents. We offer recommendations to optimise data accuracy and highlight challenges inherent to the collection of data via questionnaires. The data will be made openly available on the RADA repository once our remaining scientific papers have been accepted for publications. The preprints and R code will appear on the OSF.

So many variables, but what causes what? An invitation to Causal Inference Modelling

 De Cat, C., & Unsworth, S. (in press). So many variables, but what causes what? Journal of Child Language. (manuscript)

This is a reply to Johanne Paradis’ keynote article (Sources of individual differences in the dual language development of heritage bilinguals, JCL), which provides a comprehensive overview of factors influencing bilingual children’s dual language abilities. In this commentary, we provide an illustration of how the complex relationships between the variables discussed in Paradis’s keynote article could be conceptualised within a causal inference approach. We offer a modest starting point by summarising key features of causal inference modelling and by illustrating how it might help us better understand what causes what.

Using Latent Class Analysis to identify multilingual family profiles

Wood, Megan, Gunning, Lydia, & De Cat, Cécile. (under review). Achieving representativity in opportunity sampling: the ‘Bradford effect’ in the multilingual families Covid-19 survey. 

Typically, families from ethnic minority backgrounds and socioeconomic disadvantage are underrepresented in research. Using secondary data from a survey of the language practices of multilingual families during the first wave of the Covid-19 pandemic, we sought to ascertain whether the large proportion of Bradford-based respondents affected the representativeness of the sample and/or the patterns of responses. Respondents were objectively categorised into five latent profiles, based on the demographics of the household and families’ engagement with the Heritage Language (HL). The three of the household profiles considered “engaged” with their HL reported more positive attitudes to multilingualism. Their language practices also more markedly changed during the Lockdown compared to families who were less engaged with their HL. One of the five profiles was deemed to represent those usually considered “Hard-to-reach” in research (i.e., ethnic minority, low socio-economic status). The “Hard-to-reach” and “Unengaged” profiles were mainly represented by Bradford families. We argue that the exceptional engagement of Bradford communities in research resulted in better representativity of family profiles in this national survey.

How to quantify bilingual experience? Findings from a Delphi consensus survey

De Cat, C., Kašćelan, D., Prévost, P., Serratrice, L., Tuller, L., Unsworth, S., & consortium, Q.-B. (2023). How to quantify bilingual experience? Findings from a Delphi consensus survey. Bilingualism: Language and Cognition, 26(1), 112-124. doi:10.1017/S1366728922000359

While most investigations of bilingualism document participants’ language background, there is an absence of consensus on how to quantify bilingualism. The high number of different language background questionnaires used by researchers and practitioners jeopardises data comparability and cross-pollination between research and practice. Using the Delphi consensus survey method, we asked 132 panellists (researchers, speech and language therapists, teachers) from 29 countries to rate 124 statements on a 5-point agreement scale. Consensus was pre-defined as 75% agreement threshold. After two survey rounds, 79% of statements reached consensus. The need for common measures to quantify bilingualism was acknowledged by 96% of respondents. Agreement was reached to document: language exposure and use, language difficulties, proficiency (when it cannot be assessed directly), education and literacy, input quality, language mixing practices, and attitudes (towards languages and language mixing). We discuss the implications of these findings for the creation of a new tool to quantify bilingual experience.

This publication is one of the outputs from the Q-BEx project.

Commentary on a special volume on Children’s Acquisition of Referentiality in Narratives

De Cat, Cécile (2022). Opportunities and challenges in the analysis of the Multilingual Assessment Instrument for Narratives (MAIN). First Language DOI:10.1177/01427237211064695. (manuscript)

Abstract: The development of the Multilingual Assessment Instrument for Narratives (MAIN) has no doubt contributed to prompting a renewed interest in children’s narratives. This carefully controlled test of narrative abilities elicits a rich set of measures spanning multiple linguistic domains and their interaction, including lexis, morphosyntax, discourse-pragmatics, as well as various aspects of narrative structure, communicative competence, and language use (such as code-switching). It is particularly well suited to the study of discourse cohesion, referential adequacy and informativeness, and of course to the study of narrative structure and richness, and the acquisition of a more formal or literary register. In this commentary article, I reflect on the five empirical papers included in the special issue. I focus on methodological challenges for the analysis of narratives and identify outstanding questions.

Open Science practices in applied linguistics: results from a preliminary survey

 With Men Liu, we have analysed and interpreted the results from the brief survey I carried out in spring 2021. These have been written up as a chapter for this forthcoming volume: Plonsky, L. (Ed.). Open Science in Applied Linguistics. John Benjamins. It can be downloaded from OSF preprints.  The data and code are also available via the /OSF.