Artigos Científicos

Translation and adaptation of the Child Posttraumatic Cognitions Inventory (cPTCI) to Brazilian Portuguese

Beatriz de Oliveira Meneguelo Lobo; et al

21 de outubro de 2014

Trends Psychiatry Psychother. vol.36 no.2 Porto Alegre abr./jun. 2014

 

Translation and adaptation of the Child Posttraumatic Cognitions Inventory (cPTCI) to Brazilian Portuguese

Tradução e adaptação do Child Posttraumatic Cognitions Inventory (cPTCI) para o português brasileiro

Beatriz de Oliveira Meneguelo  Lobo 1 , Alice Einloft  Brunnet 2 , Thiago Loreto Garcia da  Silva 3 , Lafaiete Moreira dos  Santos 4 , Gustavo  Gauer 5 , Adriane Xavier  Arteche 6 , Christian Haag  Kristensen 7

1Psychologist. MSc candidate in Human Cognition, Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, RS, Brazil

2Psychology student, PUCRS, Porto Alegre, RS, Brazil

3Psychologist, PUCRS, Porto Alegre, RS, Brazil

4Psychologist. MSc in Molecular Medicine, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, MG, Brazil. Staff, Laboratory of Neuropsychological Investigations, National Science and Technology Institute for Translational Medicine, UFMG, Belo Horizonte, MG, Brazil

5Psychologist. MSc and PhD in Developmental Psychology. Adjunct professor, Department of Developmental and Personality Psychology, Institute of Psychology, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil

6Psychologist. Professor, Graduate Program in Human Cognition, PUCRS. MSc and PhD, UFRGS, Porto Alegre, RS, Brazil

7Psychologist, PUCRS. MSc and PhD in Developmental Psychology, UFGRS, Porto Alegre, RS, Brazil. Head and adjunct professor, Graduate Program in Psychology, head, Research Group on Cognition, Emotion, and Behavior, and adjunct head, Study and Research Group in Trauma and Stress, PUCRS, Porto Alegre, RS, Brazil

ABSTRACT

Introduction:

Experiencing a traumatic event is a risk factor for the development of mental illness, especially posttraumatic stress disorder. A child's appraisals of a traumatic event may play a prominent role in the development or maintenance of the disorder. Therefore, subjective responses should be evaluated to understand the impact of a traumatic event on a child's life. This study translated and adapted the Child Posttraumatic Cognitions Inventory (cPTCI) for use in linguistic and cultural contexts in Brazil.

Methods:

The process included translation, back-translation, language expert evaluation and expert committee's evaluation.

Results:

Content validity index scores were good for all dimensions after evaluation by two judges and one reformulation. The back-translation of the final version also showed that the cPTCI items in Brazilian Portuguese maintained the same meanings of the original in English. This version was tested in a sample of the target population, and all the items were above the cut-off point (minimum = 3.6; maximum = 4.0).

Conclusions:

This study was successful in producing a Brazilian version of the cPTCI. Further studies are underway to examine the reliability and the factorial and concurrent validity of cPTCI subscales.

Key words: Transcultural adaptation; PTSD; child; Child Posttraumatic Cognitions Inventory

RESUMO

Introdução:

Eventos traumáticos são fator de risco para o desenvolvimento de transtornos mentais, especialmente o transtorno do estresse pós-traumático. As avaliações que a criança faz do evento traumático podem ter um papel importante no desenvolvimento ou manutenção desse transtorno, e as respostas subjetivas dadas pela criança devem ser avaliadas para que se possa entender o impacto de tal evento em sua vida. Este estudo traduziu e adaptou o Child Posttraumatic Cognitions Inventory (cPTCI) para uso em contextos linguísticos e culturais brasileiros.

Método:

Os procedimentos incluíram a tradução, retrotradução, avaliação por especialista em linguística e avaliação por uma comissão de especialistas.

Resultados:

Os resultados do índice de validade de conteúdo foram bons para todas as dimensões após a avaliação de dois especialistas e uma reformulação. A retrotradução da versão final também demonstrou que a versão em português brasileiro do cPTCI manteve os mesmos significados da versão original em inglês. A versão final foi testada na população alvo, e todos os itens ficaram acima do ponto de corte (mínimo = 3,6; máximo = 4,0).

Conclusões:

Este estudo produziu uma versão brasileira do cPTCI. A confiabilidade, a validade fatorial e validade concorrente das subescalas do cPTCI estão sendo avaliadas por estudos em desenvolvimento.

Palavras-Chave: adaptação transcultural; TSPT; criança; cPTCI

INTRODUCTION

An estimated 25% of all children will experience a potentially traumatic event before they are 16 years old.1 Such events are risk factors for the development of mental illnesses, especially posttraumatic stress disorder (PTSD).2Epidemiological studies have found different prevalence rates of PTSD among children, ranging from 0.4% to 9% in individuals exposed to a traumatic event.3 , 4 These rates indicate that traumatic events are not the only factors, but that they may be the most important among those associated with the development of PTSD.2

Cognitive models of PTSD indicate that trauma appraisals may play a prominent role in the development or maintenance of this disorder.5 The same models may also apply to children and predict the development of PTSD in this population.6 Therefore, subjective responses should be evaluated to understand how and to what extent traumatic events affect a child's life.7

The Child Posttraumatic Cognitions Inventory (cPTCI),8 an adaptation of the Posttraumatic Cognitions Inventory (PTCI)9 , 10 for children and adolescents, assesses negative posttraumatic appraisals of events in this population. The cPTCI is intended to provide information about a potentially significant mechanism in the development of PTSD in children and adolescents, as well as to be a clinically useful tool for the assessment and prediction of this disorder.8 Our study included a sample of children and adolescents aged 6-17 years. The first group (G1) was composed of children and adolescents participating in a cross-sectional community study and who had not experienced any traumatic event. The second group (G2) comprised children and adolescents who were victims of a traumatic event. The third group (G3) was composed of participants in a study of children and adolescents who had been hospitalized because of various types of injury.

The cPTCI has two main components, described as a sense of 'permanent and disturbing change' (PC) and a sense of being a 'fragile person in a scary world' (SW).8 The original study showed that the scale has good internal consistency. Convergent validity was examined by associating each sub-scale and total score with two posttraumatic stress self-report measures (Revised Impact of Event Scale, child version [RIES-C]; Child Posttraumatic Stress Scale [CPSS]). Both sub-scales and total score were significantly and positively correlated with RIES-C and CPSS, as well as with the Depression Self-Rating Scale (DSRS), a measure of depressive symptoms. Discriminant validity was assessed by comparing children and adolescents with and without PTSD. Participants with PTSD and acute stress disorder (ASD) scored higher on cPTCI sub-scales and had a higher total score than the those without PTSD.8

Despite the relative diversity of PTSD instruments in the international literature, there is a clear lack of measures to assess posttraumatic symptoms associated with childhood trauma in Brazil.11 To date, no valid measures to evaluate posttraumatic cognitions among children in Brazil have been published. This study translated and adapted the cPTCI for use in linguistic and cultural contexts in Brazil.

METHOD

The translation and adaptation of the cPTCI were based on guidelines of the International Test Commission and previous literature12 - 16 and followed five steps: 1) translation and back-translation; 2) expert committee's evaluation; 3) pretest in target population; and 4) final back-translation and author's evaluation.

This study was approved by the Ethics in Research Committee of the institution where it was conducted (protocol no. 247.127).

Translation and back-translation

Initially, the original version was translated into Brazilian Portuguese by two translators fluent in English and with a good understanding of Brazilian Portuguese. The two translated versions were combined, and this version was back-translated independently by two Brazilian Portuguese native speakers fluent in English. The two back-translated versions were again combined.

Expert committee's evaluation

The expert committee's evaluation was conducted in two steps. The first was a meeting with a language specialist and the authors of the study. The translation and back-translation versions were compared with the original to certify that all items expressed the same ideas in both the original and the adapted versions of the questionnaire and to achieve semantic, idiomatic, experiential and conceptual equivalence between the source and the version. This evaluation generated the adapted version.

In the second step, the adapted version was evaluated by an expert committee, composed of three judges: one psychiatrist (MD, PhD) with expertise in child trauma, one psychologist (PhD) with expertise in PTSD, and one psychologist (PhD) with expertise in child psychological assessment. The expert committee assessed the content validity index (CVI) according to ratings of the relevance of each item on a 5-point Likert-like scale.15 CVI was calculated according to each judge's evaluations of three dimensions: a) language clarity, which measures how comprehensible the items are to the target population; b) practical relevance, which measures how adequate each item is to evaluate the target population; and c) theoretical relevance, which measures how much the items are in agreement with the construct theory.17 The items with a CVI lower than 0.818 were changed and evaluated again by the same judges. This step generated the pre-final version.

Pretest in target population

The pre-final version was then assessed by applying it to a sample of the target population in a pilot study. The understanding of each scale item was evaluated using a four-point verbal-numeric scale. The sample was composed of ten children and adolescents (four boys) aged 8 to 15 years (M =12.78; SD = 2.33) recruited from public schools in the metropolitan area of Porto Alegre, Brazil. All children and parents involved in this step of the process signed an informed consent form. The indicators of understanding were the scores of central tendency (mean) and dispersion (standard deviation) of the verbal rating scale for each item. Satisfactory understanding was defined as a mean score equal to or greater than three, a cut-off point based on previous studies.10 , 19 , 20

Final back-translation and author's evaluation

The final Portuguese version was back-translated by a bilingual translator, blinded to the original instrument, and the result was compared with the original version and submitted to the author's evaluation and approval.

RESULTS

The language specialist in the expert committee suggested changes in five items (2, 8, 9, 10 and 14) because of differences between the original version and the back-translation. Almost all differences occurred because the translators changed grammar structures (e.g., verbal tenses) or used idiomatic expressions that are not adequate in Portuguese when literally translated (e.g., "I am no good").

Results of the first evaluation by the expert judges showed that several items still needed to be changed. The final validity coefficient (CVIf) of language clarity was also below the cut-off point, as 11 items (1, 4, 6, 7, 9, 13, 16, 17, 19 and 20) were poorly rated (Table 1). Two items (2 and 9) were below the cut-off point on theory relevance, but the final rates (CVIf) of theory relevance and pertinence in practice were adequate, and further reformulations were focused on the language clarity dimension. All items below the cut-off point were rewritten according to the judges' suggestions and then reevaluated by the same committee. The scores of the second evaluation are shown in Table 1.

 

Table 1  CVI results of evaluation by judges 

Item Original version First evaluation CL p-value TR Second evaluation CL p-value TR
1 Anyone could hurt me. Qualquer pessoa poderia me machucar. 0.763* 0.963 0.963 Qualquer pessoa pode me fazer mal. 0.963 0.963 0.963
2 Everyone lets me down. Todo mundo me deixa triste. 0.895 0.895 0.763*        
3 I am a coward. Eu sou um covarde. 0.963 0.963 0.895        
4 My reactions since the frightening event mean I have changed for the worse. Minhas reações desde o evento assustador significam que eu mudei para pior. 0.629* 0.963 0.963 Minhas reações desde a situação ruim significam que eu mudei para pior. 0.895 0.895 0.895
5 I don’t trust people. Eu não confio nas pessoas. 0.963 0.963 0.963        
6 My reactions since the frightening event mean something is seriously wrong with me. Minhas reações desde o evento assustador significam que alguma coisa está seriamente errada comigo. 0.693* 0.963 0.963 Minhas reações desde a situação ruim significam que alguma coisa está seriamente errada comigo. 0.829 0.895 0.895
7 I am no good. Eu não sou bom. 0.696* 0.895 0.895 Eu não sou uma pessoa boa. 0.895 0.829 0.829
8 Not being able to get over all my fears means that I am a failure. Não conseguir superar todos os meus medos significa que eu sou um fracasso. 0.829 0.963 0.963        
9 Small things upset me. Pequenas coisas me incomodam. 0.763* 0.895 0.763* Mesmo coisas sem importância me incomodam. 0.963 0.829 0.829
10 I can’t cope when things get tough. Não consigo lidar quando as coisas ficam difíceis 0.533* 0.895 0.895 Não sei o que fazer quando as coisas ficam difíceis. 0.963 0.895 0.895
11 I can’t stop bad things from happening to me. Eu não consigo impedir que coisas ruins aconteçam comigo. 0.866 0.963 0.963        
12 I have to watch out for danger all the time. Eu tenho que tomar cuidado com o perigo o tempo todo. 0.895 0.963 0.963        
13 My reactions since the frightening event mean I will never get over it. Minhas reações desde o evento assustador significam que nunca irei superá-lo. 0.629* 0.963 0.963 Minhas reações desde a situação ruim mostram que nunca irei superá-lo. 0.895 0.963 0.963
14 I used to be a happy person but now I am always sad. Eu costumava ser uma pessoa feliz mas agora estou sempre triste. 0.963 0.963 0.963        
15 Bad things always happen. Coisas ruins sempre acontecem. 0.829 0.963 0.963        
16 I will never be able to have normal feelings again. Nunca vou conseguir ter sentimentos normais de novo. 0.763* 0.829 0.829 Nunca vou conseguir ter os mesmos sentimentos que eu tinha antes. 0.829 0.895 0.829
17 I’m scared that I’ll get so angry that I’ll break something or hurt someone. Tenho medo de ficar com tanta raiva que quebrarei alguma coisa ou machucarei alguém. 0.696* 0.895 0.895 Tenho medo de ficar com tanta raiva e acabar quebrando alguma coisa ou machucando alguém. 0.895 0.895 0.895
18 Life is not fair. A vida não é justa. 0.963 0.963 0.963        
19 My life has been destroyed by the frightening event. Minha vida foi destruída pelo evento assustador. 0.763* 0.963 0.963 Minha vida foi destruída pela situação ruim. 0.963 0.963 0.963
20 I feel like I am a different person since the frightening event. Eu sinto que sou uma pessoa diferente desde o evento assustador. 0.763* 0.963 0.963 Eu sinto que sou uma pessoa diferente desde a situação ruim. 0.963 0.963 0.963
21 My reactions since the frightening event show that I must be going crazy.   0.732 1 1 Minhas reações desde a situação ruim mostram que devo estar ficando louco(a). 0.933 1 1
22 Nothing good can happen to me anymore. Nunca mais vão acontecer coisas boas comigo. 1 1 1        
23 Something terrible will happen if I do not try to control my thoughts about the frightening event.   0.6 0.866 0.866 Alguma coisa terrível vai acontecer se eu não tentar controlar meus pensamentos sobre a situação ruim. 1 0.933 0.933
24 The frightening event has changed me forever.   0.6 0.732 1 A situação ruim me mudou para sempre. 0.933 0.866 0.963
25 I have to be really careful because something bad could happen.   1 1 1 Eu preciso ser muito cuidadoso(a) porque alguma coisa ruim pode acontecer.      
  CVCf   0.778* 0.927 0.923   0.914 0.927 0.919

CL

= clarity of language

CVI

= content validity index

TR

= theoretical relevance

*

Only the items below the cut-off point were reevaluated.

 

The judges pointed that there is no expression in Portuguese that directly replaces "frightening event" while maintaining the inherent meanings of fear and threat found in the original. Therefore, they suggested an alternative term for the expression "bad situation." The other changes were made to improve comprehension and clarity for children and adolescents (e.g., the Portuguese equivalent of the English word "cope" is an unusual expression and can be hard for children to understand) and to maintain the same original meaning of some words (e.g., if literally translated, the English word "hurt" means only physical injury in Portuguese).

The second evaluation by the judges revealed that the items no longer had a CVI bellow the cut-off point after changes. This version was tested in the target population and all the items were above the cut-off point (M = 3.94; SD = 0.11).

After those terms were changed, the final Portuguese version was back-translated. The comparison is shown inTable 2.

 

Table 2  Back-translation of the final version 

Item Original version Final version
1 Anyone could hurt me. Anyone could hurt me.
2 Everyone lets me down. Everyone makes me sad.
3 I am a coward. I’m a coward.
4 My reactions since the frightening event mean I have changed for the worse. My reactions since the bad situation mean that I changed for worse.
5 I don’t trust people. I don’t trust people.
6 My reactions since the frightening event mean something is seriously wrong with me. My reactions since the bad situation mean that something is hardly wrong with me.
7 I am no good. I’m not a good person.
8 Not being able to get over all my fears means that I am a failure. Not being able to get over all my fears means that I’m a looser.
9 Small things upset me. Even things without importance bother me.
10 I can’t cope when things get tough. I don’t know what to do when things get difficult.
11 I can’t stop bad things from happening to me. I’m not able to prevent bad things happening to me.
12 I have to watch out for danger all the time. I need to take care with danger all the time.
13 My reactions since the frightening event mean I will never get over it. My reactions since the bad situation show that I am never going to overcome it.
14 I used to be a happy person but now I am always sad. I used to be a happy person, but now I’m always sad.
15 Bad things always happen. Bad things always happen.
16 I will never be able to have normal feelings again. I’m never going to have the same feelings that I had before.
17 I’m scared that I’ll get so angry that I’ll break something or hurt someone. I’m afraid of staying so angry that I could end up breaking or hurting something or someone.
18 Life is not fair. Life is not fair.
19 My life has been destroyed by the frightening event. My life was destroyed by the bad situation.
20 I feel like I am a different person since the frightening event. I feel that I’m a different person since the bad situation.
21 My reactions since the frightening event show that I must be going crazy. My reactions since the bad situation show that I must be going crazy.
22 Nothing good can happen to me anymore. Good things are never going to happen to me anymore.
23 Something terrible will happen if I do not try to control my thoughts about the frightening event. Something terrible is going to happen if I don’t try to control my thoughts about the bad situation.
24 The frightening event has changed me forever. The bad situation changed me forever.
25 I have to be really careful because something bad could happen. I need to be very careful because something bad can happen.

 

Table 2 shows that the changes of the Portuguese version did not affect the theoretical construct latent in the original items. Moreover, most items had exactly the same expressions and words in the final back-translation.

DISCUSSION

This study was conducted to translate and cross-culturally adapt the cPTCI to the Brazilian context. Studies in the literature suggest that there is no universal agreement on how to adapt an instrument for use in another cultural setting, but an adequate cross-cultural adaptation process is crucial to reducing bias in a study.13 , 21 Given the lack of a criterion standard for translation and cultural adaptation, this study adopted the steps recommended in the literature: translation, back-translation, linguistic and cultural adaptation and expert committee's evaluation.12 , 13 , 15 , 16 , 22

CVI scores for all dimensions were good according to the expert committees' evaluations. The back-translation of the final version also confirmed that the Portuguese cPTCI maintained the same meanings of the original in English. That compatibility is an important indicator of cross-cultural validity in studies about posttraumatic reactions among children.21 , 23

The adaptation of the language used in a psychological instrument for children and adolescents was one of the difficulties faced in this study. The instrument was developed for a sample of individuals aged 6 to 17 years, and there are considerable differences in knowledge of words and expressions between children and adolescents. Therefore, the pilot study was fundamental to make sure that children of different ages were able to understand all items.

Most psychometric questionnaires are devised and originally developed in English-speaking countries. However, researchers and practitioners will often be faced with the need to use these instruments to assess populations that do not have English as their native language. Translations should be conducted with methodological rigor and should include cross-cultural adaptation that addresses both linguistic and cultural issues.12 , 22 , 23 These methods must include objective measures of content validity to ensure the quality of the instrument.

One of the strengths of this study was the use of the CVI for the expert committee's evaluation. This index provided quantitative data to evaluate the adequacy of the adapted cPTCI items and to define objective reformulations focusing specifically on clarity of language, practice pertinence and theory relevance. Moreover, pilot testing in the target population was important to identifying confusing and misleading connotations in several items. Further studies are underway to examine the reliability and the factorial and concurrent validity of the cPTCI.

ACKNOWLEDGEMENTS

This work was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq).

 

REFERENCES

 

1. Costello EJ, Erkanli A, Fairbank JA, Angold A. The prevalence of potentially traumatic events in childhood and adolescence. J Trauma Stress. 2002;15:99-112.

2. Trickey D, Siddaway AP, Meiser-Stedman R, Serpell L, Field AP. A meta-analysis of risk factors for posttraumatic stress disorder in children and adolescents. Clin Psychol Rev. 2012;32:122-38. Epub 2011 Dec 8.

3. Elklit A. Victimization and PTSD in a Danish national youth probability sample. J Am Acad Child Adolesc Psychiatry. 2002;41:174-81.

4. Meltzer H, Gatward R, Goodman R, Ford T. Mental health of children and adolescents in Great Britain. London: The Stationery Office; 2000.

5. Ehlers A, Clark DM. A cognitive model of posttraumatic stress disorder. Behav Res Ther. 2000;38:319-45.

6. Ehlers A, Mayou RA, Bryant B. Cognitive predictors of posttraumatic stress disorder in children: results of a prospective longitudinal study. Behav Res Ther. 2003;41:1-10.

7. Margolin G, Vickerman K. Post-traumatic Stress in Children and Adolescents Exposed to Family Violence: I. Overview and Issues. Prof Psychol Res Pr. 2007;38(6):613-19.

8. Meiser-Stedman R, Smith P, Bryant R, Salmon K, Yule W, Dalgleish T, et al. Development and validation of the Child Post-Traumatic Cognitions Inventory (CPTCI). J Child Psychol Psychiatry. 2009;50:432-40.

9. Foa EB, Tolin DF, Ehlers A, Clark DM, Orsillo SM. The Posttraumatic Cognitions Inventory (PTCI): development and validation. Psychol Assess.1999;11:303-14.

10. Sbardelloto G, Schaefer LS, Justo AR, Lobo Bde O, Kristensen CH. Adaptation and content validation of the Brazilian version of the Posttraumatic Cognitions Inventory. Rev Saude Publica. 2013;47:326-34.

11. Borges JL, Zoltowski APC, Zucatti APN, Dell'Aglio DD, Transtorno de estresse pós-traumático (TEPT) na infância e na adolescência: prevalência, diagnóstico e Avaliação. Aval Psicol. 2010;9:87-98.

12. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25:3186-91.

13. Gjersing L, Caplehorn JR, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations. BMC Med Res Methodol. 2010;10:13.

15. Hernández-Nieto RA. Contribuciones al análisis estadístico. Mérida, Venezuela: Universidad de Los Andes/IESINFO; 2002.

14. Herdman M, Fox-Rushby J, Badia X. A model of equivalence in the cultural adaptation of HRQoL instruments: the universalist approach. Qual Life Res. 1998;7:323-35.

16. International Test Commission [Internet]. International Test Commission guidelines for translating and adapting tests. 2010 [cited 2013 nov 18]. http://www.intestcom.org/upload/sitefiles/40.pdf 

17. Balbinotti MA, Benetti C, Terra PRS. Translation and validation of the Graham-Harvey survey for the Brazilian context. Int J Manage Financ. 2007;3:26-48.

18. Clark P, Lavielle P, Martinez H, Learning from pain scales: patient perspective. J Rheumatol. 2003;30:1584-8.

19. Grassi-Oliveira R, Stein LM, Pezzi JC. [Translation and content validation of the Childhood Trauma Questionnaire into Portuguese language]. Rev Saude Publica. 2006;40:249-55. Epub 2006 Mar 29.

20. Oliveira SES, Bandeira DR. Linguistic and cultural adaptation of the Inventory of Personality Organization (IPO) for the Brazilian culture. J Depress Anxiety. 2011;1:1-7.

21. Bender HA, Martín García A, Barr WB. An interdisciplinary approach to neuropsychological test construction: perspectives from translation studies. J Int Neuropsychol Soc. 2010;16:227-32. Epub 2010 Jan 29.

22. de Silva HA, Gunatilake SB. Mini Mental State Examination in Sinhalese: a sensitive test to screen for dementia in Sri Lanka. Int J Geriatr Psychiatry. 2002;17:134-9.

23. Cha ES, Kim KH, Erlen JA. Translation of scales in cross-cultural research: issues and techniques. J Adv Nurs. 2007;58:386-95. Epub 2007 Apr 17.

The present study was carried out at the Graduate Program in Psychology, Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, RS, Brazil.

Financial support: Beatriz de Oliveira Meneguelo Lobo and Christian Haag Kristensen are recipients of scholarships from Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq).

 

No conflicts of interest declared concerning the publication of this article.


Artigo original:

http://www.scielo.br/scielo.php?script=sci_arttext&pid=S2237-60892014000200107&lng=pt&nrm=iso&tlng=en

O portal Psiquiatria Infantil.com.br já recebeu

49.739.541 visitantes