INTRODUCTION

Undergraduate research experiences may motivate students to learn about scientific method, encourage them to follow academic careers, post- graduate studies and provide other personal benefits (^{Lopatto, 2004}). The participation of undergraduate students as study subjects (SS) in research is considered a fundamental research activity which may motivate or improve learning the scientific field and generate social opportunities (^{Brewer & Robinson, 2018}). Growing demand for a university education means that the participation of students as SS will be more frequent, and it is fundamental to create an enriching educational opportunity with proper measurement instruments (^{Calderon, 2018}). The scarce empirical evidence available focuses mainly on determining, from the students' point of view, the educational advantage of this activity, namely reinforcement of their knowledge of scientific method; some also consider it an important opportunity for learning directly about how to carry out research and experimentation (^{Forester & McWhorter, 2005}; ^{Moyer & Franklin, 2011}; ^{Roberts & Allen, 2013}). Apart from the educational benefit, the ethical aspects involved in participation are also important; they must be correctly identified, always ensuring an independent, voluntary decision free of any undue or coercive influence on the students (^{Boileau et al., 2017}). Greater information about student participation as SS allows the teaching impact of this activity to be reinforced, complementing the ethical evaluations of review institutions (Roberts & Allen; Brewer & Robinson). However, identifying the right indicators for measurement is complicated and there are still only a limited number of solid psychometric instruments available (^{Leentjens & Levenson, 2013}).

The evidence suggests starting by identifying the participation of students as SS according to the principles of learning (^{Healey & Jenkins, 2009}; ^{Ambrose et al., 2010}). This allows educators to recognize the students' prior knowledge gained in other courses or during their everyday life, represented by facts, concepts, models, perceptions, beliefs, values, attitudes or other ways in which humans interact with their surroundings. They can then understand how the students filter and interpret the information, and whether this represents an advantage or an obstacle to the acquisition of greater knowledge (Ambrose et al.). This will allow them to design appropriate teaching which will promote learning. Some students of the medical and psychological sciences have considered educational research fundamental for their professional skills, important for their future interactions with the scientific world as students, and a basic item of knowledge (Forester & McWhorter; ^{Sarpel et al., 2013}; Brewer & Robinson). In this area, a questionnaire that determine the students' perception of the educational benefit of participation stands out, which can be used for different research purposes (Roberts & Allen).

Learning is also modulated by motivation, and the drivers of learning are goals; they provide the context in which values and expectancy become meaningful (Ambrose et al.). The importance attached to attaining a goal is defined as subjective value, or simply value, and is influenced by interest in a particular content. Developing interest for a subject depends on the interaction between an enriching situation and other individual aspects. It is here that a positive experience during learning allows students to generate greater interest, and therefore a high value and certainly better expectations of the experience and content of interest (^{Hidi et al., 2004}). Participation in research as a SS should be considered an interesting experience and not a frustrating situation.

In one work, medical students who considered educational research to be fundamental in the medical sciences gave a contradictory reply when asked if they were interested in participating as SS. The authors gave two possible explanations: the students believed that lack of time would prevent them from participating, or they did not understand the expectancies of participating as SS (Forester & McWhorter). Some attach great value to an altruistic desire to contribute to science, or to the learning involved in the experience, expressing less concern about the extra time invested in participation; while others recognize that the lessons learned will contribute to the development of their own research projects or to the knowledge of future generations of students (Sarpel et al.; Brewer & Robinson). It is important not to create frustrating or stressful situations which may make the participants uncomfortable, and which, combined with certain personal characteristics and incorrect perception of the time involved, may lead to a poor appreciation of the experience (Brewer & Robinson).

However, the value attached to a content is not sufficient to motivate behaviour. It is also necessary to believe that the goal can be successfully achieved (Ambrose et al.). These are outcome expectancies and also influence how the student applies different behaviors to pursue success. These behaviors are influenced by the individual's belief that he/she is capable of achieving the objective, and by his prior experience with the content, enabling him to identify the reasons which will or would enable him to achieve the goal. The presence of positive or negative influences generates positive or negative expectancies of the activity, which modulate motivation. Identifying these, allows educational strategies to be designed which are capable of strengthening students' motivation to learn and their whole professional development - including scientific thinking (Ambrose et al.; Sarpel et al.). On the other hand, undue influence and coercion must be avoided, as they diminish the voluntary nature of the decision to participate as a SS, especially when the students are in a dependent relationship with the teacher(s) - investigators (Sarpel et al.; Council for International Organizations of Medical Sciences, 2016). However, the evidence that exists has shown little presence of such undue influences in medical students either in USA (Forester & McWhorter; Sarpel et al.) or India (^{Vaidya et al., 2016}) and shows the clear difference in the response between different populations.

After this analysis, we observed a larger number of studies in the social sciences and psychology, and a smaller number in health sciences where academic research is carried out constantly. Evidence on this experience of participation measured in Spanish- speaking populations is even scarcer (^{Astete et al., 2019}). However, we considered that there was sufficient evidence available to develop an instrument with which to explore, from the perspective of the participants, how academic research has been carried out when students have been SS, and their opinion of this activity. This encouraged us to create a multidimensional questionnaire to measure the “experience of participation by undergraduate students as study subjects”, our definition of the hypothetical construct. As it represents a concept which is not easy to define, it must be measured through an index using different methods (^{Streiner, 2003}). The questionnaire was designed according to the criteria for the creation of measurement scales (Streiner et al., 2015). The sources of information for design of the items were theories, previous scales and the opinion of experts. We considered that it was fundamental to discover the student's perception of the educational benefit of participating as a SS for his professional development, describing his interest in participating and describing the factors which motivated students to decide to participate. The questionnaire was called “Participation of Undergraduate Students as Study Subjects Index, USTUDI”. The perception of the educational benefit, interest and influences in favor of participation as SS represented the most important attributes for university students in this experience, according to the literature consulted; these are used as the study variables. The purpose of this research was to design this new index and evaluate the validity of its content based on the judgement of experts.

MATERIAL AND METHOD

Item selection and design. Figure 1 shows how the study variables relate to the construct. We defined perception as the only indicator of effect, and interest and influences as cause indicators. The importance of these latter is in the specific type of item chosen and not the correlation of these with a factorial theorem (^{Bollen & Lennox, 1991}; Streiner).

Fig. 1 Diagram of effect (a) and causal (b) indicators of the construct «Participation of undergraduate students as study subjects». 

Operationalization of variables was doing to identifying their conceptual definition, operational definition and the dimensions of the variables, indicators, type of item and total items per variable (Table I).

The socio-demographic items included in the questionnaire are sex, date of birth, curricular progress and previous professional training. These variables were selected according to the literature consulted, which reported interesting results of the analyses (Lopatto; Forester & McWhorter; Roberts & Allen; Vaidya et al.). Because the participants represent a vulnerable population (Council for International Organizations of Medical Sciences), their confidentiality was respected by coding the questionnaires. Four filter questions were drafted to guide the respondents through the questionnaire, and detect who had accepted, refused or abandoned participations; and in the case of those who had participated at some time, what their experience as SS had been.

With the help of experts in education and health, we translated and adapted two scales consulted in the literature which measure the indicators "value attached to the educational benefit of having participated as SS" (Item 10, Table I) and "undue coercive influences in the decision of university students to participate as SS in research” (Item 9, Table I). No scale was found which measured the interest of students in this research activity. Permission was obtained from one of the authors to translate and adapt the original Student Perceptions of the Educational Value of Research Participation Scale (SPEVRPS) and the Coercion Assessment Scale into Spanish. The original questionnaires were translated from English independently by two native speakers of Chilean Spanish, and then translated back by two native speakers of English who were familiar with Spanish and did not know the purpose of the research. To resolve the discrepancies between the versions and guarantee equivalence of the concepts, the two versions were reviewed by a mixed committee of other native speakers of both languages, together with the panel of experts and the authors of this study. The conceptual, semantic and operational equivalence and the equivalence by item of the translations of the scales were confirmed, resulting in a questionnaire allowing versatile use of formats, instructions or mode of administration (^{Herdman et al., 1998}; Streiner et al.). No important conceptual differences were observed between the two versions or in evaluation of the equivalence of the items. However, one change was made to one of the items which used the verb “feel” in English = “sentir” in Spanish, in the past or present tense; this was replaced by a more objective verb: “pensar” in Spanish = "think" in English (“lamenté o pensé”) to obtain semantic equivalence. The administration of the scales presented no operational problems. Both scales are for exclusive use with students who have participated previously in research as SS.

Table I Operationalization of the study variables. 

The indicator “Educational benefit obtained by the student from participation” (Item 12) is a mixed multiple choice item, since this dimension is considered to be undergoing exploration. The main purpose of this item is to identify students who do not recognize an educational benefit of participation, and other perceived characteristics of this activity.

The indicator “interest in participation recognized by the student” (Item 11) was represented in a categoric item with answers: not interesting, little interest, moderate interest and very interesting, with no ambivalent neutral term. We decided on this continuous scale to identify the presence or absence of interest, and its degree where interest existed.

The dimension “lasting disposition towards a particular content and on repeated occasions” is represented by two indicators: in one the student is asked to recall the number of participations as SS per academic year during his university course (Item 5); the other item is a category where students who have participated as SS in a research work can evaluate their experience from very bad to very good with an ambivalent neutral term (Item 7).

Measurement of the dimension “Factor which motivated you to accept, refuse and abandon” is represented by three indicators, each with one item, in which having accepted to participate in a research work leads to a mixed ranking question where the student orders a series of motives that might have made him agree to participate as a SS in order of priority from 1 (most important) to 7. The purpose of this item is to identify the principal factor which guided his behavior, which he can select or write accordingly (Item 13). The other two items, one to investigate the reason for refusing to participate and the other for abandoning the study, are mixed multiple choice questions (Items 15 & 17).

Validity based on judgement of experts. Once the SPEVRPS and the Coercion Assessment Scale had been translated, the cover and relevance of the items were evaluated to ensure content validity. As Tables II and III show, to ensure correct cover of the domains, they must be represented by a minimum of three items, and each item must be included in only one domain (Streiner et al.). However, when determining the factorial structure of the Coercion Assessment Scale, we observed that one of its two identifiable sub-dimensions, where voluntary factors guide behavior, did not present proper cover as it was represented by only two items (Table II). In order to cover the sub-dimension correctly, it was decided to draft three more statements for this voluntary factor, in accordance with Sarpel et al. and Vaidya et al.: one item to identify whether the decision was motivated by a desire to obtain the answer to a health examination or test; one statement referring to scientific interest in participating and contributing to science; and one statement linked with an economic interest in participating as a SS. The items were kept as short and simple as possible, between 10 and 20 words, not counting the instructions on how to answer the scales.

Table II Checking validity of content for cover of Coercion Assessment Scale. 

Table III Checking content validity for cover of the SPEVRPS. 

Content validity by relevance. Four experts evaluated each item by relevance on a scale with four categories: 1= irrelevant, 2= hardly relevant, 3= relevant, 4= very relevant. The content validity indices (CVI) were calculated both for each item (I-CVI) and for the scales (S-CVI/mean), together with the modified Cohen's kappa index (k*) which considers random risk. The I-CVI is interpreted as the number of experts who awarded relevance of 3 or 4 divided by the total number of experts. The S-CVI is the mean of all the I-CVI of all the items in the scale. It was decided to maintain those items for which the I-CVI was equal to or greater than 0.75. A value of 0.90 of the S-CVI/mean indicates that the scale does not require modification; this value is considered excellent content validity. The k* value of the scales must be interpreted as: values between 0.40 and 0.59 - low correlation in the responses of the experts on the relevance of the scale; values between 0.60 and 0.74 - adequate correlation; and values higher than 0.74 - excellent correlation (^{Polit et al., 2007}). The data were stored and processed statistically in the Numbers program, version 6.2.

Table IV Checking content validity by cover of USTUDI. 

RESULTS

Content validity based on the agreement of the experts.

SPEVRPS scales.

Content validity. As Table V shows, none of the items presented a value below 0.75 in the I-CVI. The S-CVI for the SPEVRPS was 0.928 and five of the items presented an excellent k* coefficient.

Table V Evaluation of the content validity of SPEVRPS. 

^{aI-CVI (content validity index by item) = number of assessments of 3 or 4 / number of experts. bPr (Probability of random agree ment) = [N/A(N-A)] x 0.5N where N = number of experts and A = number of agreements with good relevance. ck* = Modified kappa by relevance: k* = (I-C VI - Pc)/(1 - Pc) dInt. = Statistical interpretation kappa = poor = K* of 0.40-0.59; adequate = K* of 0.60-0.74; and excellent = K* = >0.74 eS-CVI/mean (Mean content validity index of the measuring scale) = mean I-CVI}.

Coercion Assessment Scale. Content validity. Table VI shows the items after translation and adaptation to be evaluated for relevance by the pa- nel of experts. Items 1-3, 5, 6, 8-10 of the scale were considered very relevant by the experts. Item 4 was assessed as hardly relevant by half of the experts and presents a very low value for the k* coefficient. The S-CVI/mean value of 0.925 was interpreted as excellent, and expected to improve after the removal of item 4.

Table VI Evaluation of the content validity of the Coercion Assessment Scale. 

^{aI-CVI (content validity index by item) = number of assessments of 3 or 4 / number of experts. bPr (Probability of random agree ment) = [N/A(N-A)] x 0.5N where N = number of experts and A = number of agreements with good relevance. ck* = Modified kappa by relevance: k* = (I-C VI - Pc)/(1 - Pc) dInt. = Statistical interpretation kappa = poor = K* of 0.40-0.59; adequate = K* of 0.60-0.74; and excellent = K* = >0.74 eS-CV I/mean (Mean content validity index of the measuring scale) = mean I-CVI}.

Content validity of USTUDI questionnaire. Discounting the socio-demographic questions, filter and the adapted versions of the SPEVRPS and Coercion Assessment Scale, the items not present in the scale presented excellent relevance in the evaluation by experts. The total content validity of the instrument is the mean of the S-CVI/means of USTUDI, SPEVRPS and the Coercion Assessment Scale. Table VIIshows excellent validity in the judgement of experts, with a total S-CVI/mean of 0.951.

Table VII Evaluation of the content validity of USTUDI. 

^{aI-CVI (content validity index by item) = number of assessments of 3 or 4 / number of experts. bPr (Probability of random agreement) = [N/A(N-A)] x 0.5N where N = number of experts and A = number of agreements with good re levance. ck* = Modified kappa by relevance: k* = (I-CVI - Pr)/(1 - Pr) dInt. = Statistical interpretation kappa = poor = K* of 0.40-0.59; adequate = K* of 0.60-0.74; and excellent = K* = >0.74 eS-CVI/mean (Mean content validity index of the measuring scale) = mean I-CVI. Total S-CVI/mean = (InC items S-CVI/meane + SPEVRPS S-CVI/meane + Coercion Assessment Scale S-CVI/meane) / 3}.

DISCUSSION

Higher education recognizes research to be fundamental in the learning of university students (Healey & Jenkins). However, their participation as SS because it is a beneficial educational activity can still not be justified given the scarce empirical evidence available (Leentjens & Levenson). Responsibility for students' learning about scientific thinking is in the hands of educators and the teaching strategies that they use to strengthen this process (Healey & Jenkins; Ambrose et al.).

To complement the evidence in order to aid comprehension of this research activity and its educational benefits, we designed an instrument called “participation of undergraduate students as study subjects index USTUDI”, which presented excellent content validity for initial psychometrics. The factorial structure of USTUDI was represented by three indicators: perception of the educational benefit of participating as SS in research, interest in the experience, and influences in the decision to take part in this scientific activity. Different items were drafted in accordance with the theoretical framework selected and the evidence available in order to cover the dimensions properly. This is the first time that the SPEVRPS and the Coercion Assessment Scale have been translated and adapted to the Spanish language with an analysis of their content validity by judgement of experts. To obtain semantic equivalence in the translations, ambiguous and personal terms, i.e. the verb “sentir" (to feel), were replaced by a more suitable term to ensure optimum response. We took care that the items of the questionnaire were drafted in simple terms with the least number of words to reduce as much as possible the bias of satisfaction responses (Streiner et al.). Three new questions were drafted for the Coercion Assessment Scale in order to cover adequately one of its sub-dimensions. Item 4 of this scale was classified as redundant, and similar to item 3; it was therefore decided to eliminate it from the scale. This enabled us to obtain a suitable factorial structure for evaluation in future studies. Our findings showed scales with excellent S-CVI/mean and total values for the questionnaire. Those items with the lowest interpretation of the k* value as Adequate required minimal modifications.

This index evaluated the research work from the point of view of the students and their participation as SS in research. In this way we hoped to generate a change in how research works are carried out with students, under teaching strategies that strengthen learning of scientific method and research activities, since these experiences are a part of higher education and will continue to be carried out in future.

We also hoped to use this index to monitor the academic research carried out in institutions of higher education, strengthen the educational aspect for university students of the experience of participating as SS, determine similarities and differences between different student populations and identify changes in their responses over time. The gender differences in responses to this questionnaire will also allow complementary information to be obtained to what is already known about the poor representation of the female sex in the sciences, and what they think about this activity (^{Holman et al., 2018}).

It is fundamental to ensure that the participation of potential participants is voluntary and confidential, since students are considered to be vulnerable individuals under the CIOMS guidelines, and it is recommended to categorize the data by age for analysis if the study population is small or potentially recognizable (Sarpel et al.). Ethics institutions play an essential and beneficial role in the evaluation of these projects, ensuring protection of the participants by assessing the potential risks of participation along with issues of confidentiality and justice (Boileau et al.). However, the evidence shows that discrepancies exist in the evaluation of projects sent to different review institutions (Sarpel et al.).

We consider it essential to determine the proportion of students who present low perception of an educational benefit, little interest and negative influences in their participation as SS in research, since these factors generate conducts which will not motivate them to learn through these scientific activities, hindering the optimum development of their knowledge.

CONCLUSIONS

This is the a multidimensional, single-application instrument designed to measure three psychological attributes of the participation of university students as SS in research, with a defined methodological design process associated with validation based on the judgement of experts, making it an instrument of high educational value. Measuring prior knowledge by the perception of the educational benefit will allow educators to better guide their teaching strategies in scientific activities. Understanding how students evaluate the experience of participating as SS will allow motivation-based teaching techniques to be strengthened. Identifying influences which lead students to decide to participate as SS will improve the way in which educational research is carried out in an ethical framework. Having an instrument which effectively captures the students' opinion of this experience of participation will allow educational institutions to take informed teaching decisions for action at macro and micro-levels (^{Brew, 2013}).