-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path17-writingresults.Rmd
49 lines (28 loc) · 8.75 KB
/
17-writingresults.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# Writing up results {#results}
In this video, we are going to give you some tips and ideas for how to write up your results for three types of readers. First, we will discuss write-ups for a scientific venue. Second, we will discuss the kind of information on your results that you can share with participating families. Third, we cover write-up of results for the benefit of the community that you sampled from.
## Scientific venue
Regarding your write-up to a scientific venue, we are mostly going to give you key information and references for the methods section.
First, in the participants section, mention not only participants and data that you are including but also all the data that was collected but excluded. Two potential reasons for exclusion that are relevant in the participants section are short and silent recordings, because they suggest the family "opted out". For instance, if all families recorded for 16h but one did so for 2h, then this may suggest they changed their mind about participating. Alternatively, recordings that have mostly silence (e.g. over 90%) may mean that the family put the recorder in a drawer, because they did not want to be recorded.
Second, in the equipment section, mention not only what device you were using but also clothing, and how the device was attached to the clothing. You need to convey that the device was attached in a tight-fitting way, to minimize noise.
Next comes the analysis section. Here, you can mention the software you used to analyse the recordings automatically, and provide links to more technical reports. Links to the technical reports to the all systems are provided in the resources of this video.
Make sure you also mention relevant studies on reliability. We have a dedicated video on this, but in a nutshell, if you are interested in broad measures, like child vocalization counts or adult word counts, then there is a meta-analysis on the accuracy of these metrics, and another that measures correlations between these measures and standardized language tests. However, please note that these references do not cover all useful metrics. For instance, if you are interested in overall vocalization counts by the key child (including speech and crying), neither of those meta-analyses can inform you on that. However, looking for studies that cite those meta-analyses will probably help you discover any newer evaluations.
If you have carried out any human or manual annotations, make sure that you provide the link to the human annotation training material. Instructions you have provided annotators will affect your data, so they should be documented in the same way as you report the make and model of your recording device. Make sure you also report on any inter-rater reliability data -- where raters may be two humans, or between a human annotator and the algorithm.
## Families
In my lab, we provide summaries with the results of the studies to the families who participated in the study, so we thought to give you some tips on how to explain results to families.
Some parents want to get their own data, so you can produce an individualized graph which shows the proportion of silence, vocalizations, etc. found in the recorder. As mentioned in the piloting video, we do not think giving families the raw data is a good idea on ethical and legal grounds, but providing a visual snapshot of the recording may be even more interesting to the parents than 10+ hours of audio!
Some parents want more, for example a comparison between their quantity of speech and the other families in the study. It is up to you (and the community you are working with) to you to decide whether it is reasonable to give them this information. In my lab we reason as follows: This method is still being developed, and we don’t have a very large, representative, and normed sample on which to base our measures. Therefore, providing parents with percentiles might generate some unneeded anxiety on parents. For example, if you have 30 children in your sample, someone is going to be the 30th, and this family may perceive these results as discouraging.
Another thing you might want to do is provide them with group results, which might be a summary of your scientific paper. You can also provide them with the link to the actual paper. This is important so that they have access to the publication that enters the scientific stream. However, we are uncertain that just giving them the paper will be useful, as it will typically contain technical language, and may be pitched for specialists. So a separate write-up is appropriate, and much faster than a publication. In such a write-up, follow the usual advice for science communication: avoid technical wording, instead using a language that is understandable by someone who doesn’t have the technical background, and cover aspects of your data that families are actually likely to care about, which you may have discovered during piloting.
## Communities
Long-form recordings are capturing people’s lives to a certain extent. In fact, if you think about it, they are not just capturing the participating families’ lives, but they are in fact sampling from the lived experiences of the communities those families belong to. So in a way, long-form recordings are taking a snapshot of the life in people in a given community.
Communities don’t necessarily have a legal identity but they might be seen as a superset of all of the families that participated in your study. So, if this sounds something that applies to the work you are doing, you can think of ways to give back to the community and not just the specific families who participated.
You can do this by for instance creating a summary of the results or short presentations of your study and what you found so far, perhaps the same one you are giving participating families. You can then distribute it to associations or other entities related to that community. Even better, you can collaborate with associations closely connected to your community in producing such a write-up. Or you could try to write a newspaper article, if you know a journalist who can help you with that. In any case, this is a great opportunity to go back to the topics that were raised as key interests by members of the community during the piloting stage.
## Resources
- Example scientific paper not using LENA: Casillas, Marisa, Penelope Brown, and Stephen C. Levinson. "Early language experience in a Tseltal Mayan village." Child Development 91.5 (2020): 1819-1835. [pdf](https://pure.mpg.de/rest/items/item_3184321_3/component/file_3184322/content?download=true)
- Example scientific paper using LENA: Ma, Y., Jonsson, L., Feng, T., Weisberg, T., Shao, T., Yao, Z., ... & Rozelle, S. (2021). Variations in the Home Language Environment and Early Language Development in Rural China. International Journal of Environmental Research and Public Health, 18(5), 2671.[link](https://www.mdpi.com/1660-4601/18/5/2671/pdf)
- Example scientific paper using LENA looking at methodology: McDonald, M., Kwon, T., Kim, H., Lee, Y., & Ko, E. S. (2021). Evaluating the Language ENvironment Analysis System for Korean. Journal of Speech, Language, and Hearing Research, 64(3), 792-808. [link](https://pubs.asha.org/doi/pdf/10.1044/2020_JSLHR-20-00489?casa_token=PwzsnBJmG6wAAAAA:WgPlROxAmjGSybWEyMhswBq8XMU8BXXrsDbbbvLfYKATktWkwnvU-lX2-lfxCVvS3Roo1XpH0wc02p5c)
- [Example information for parents (French)](https://osf.io/4grsn/)
- [LENA reliability original report](https://www.lena.org/wp-content/uploads/2016/07/LTR-05-2_Reliability.pdf)
- LENA Meta-analysis on accuracy: Cristia, A., Bulgarelli, F., & Bergelson, E. (2020). Accuracy of the language environment analysis system segmentation and metrics: A systematic review. Journal of Speech, Language, and Hearing Research, 63(4), 1093-1105. [pdf](https://files.de-1.osf.io/v1/resources/fhs57/providers/osfstorage/5d9d975ca7bc73000ce8339f?direct=&mode=render)
- LENA Meta-analysis on metrics and standardized tests: Wang, Y., Williams, R., Dilley, L., & Houston, D. M. (2020). A meta-analysis of the predictability of LENA™ automated measures for child language development. Developmental Review, 57, 100921. [link](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7337141/)
- VTC Technical report: Lavechin, M., Bousbib, R., Bredin, H., Dupoux, E., & Cristia, A. (2020). An open-source voice type classifier for child-centered daylong recordings. Interspeech. [pdf](https://arxiv.org/pdf/2005.12656) [code](https://github.com/MarvinLvn/voice-type-classifier)
- ALICE Technical report: Räsänen, O., Seshadri, S., Lavechin, M., Cristia, A., & Casillas, M. (2020). ALICE: An open-source tool for automatic measurement of phoneme, syllable, and word counts from child-centered daylong recordings. Behavior Research Methods. [pdf](https://psyarxiv.com/p95dz/download?format=pdf) [code](https://github.com/orasanen/ALICE)