Effectiveness of Student Learning during Experimental Work in Primary School

The aim of the research was to optimize the effectiveness of student learning based on experimental work in chemistry classes in Slovenian primary schools. To obtain evidence about how experimental work is implemented during regular chemistry classes, experimental work was videotaped during 19 units of chemistry lessons at 12 Slovenian primary schools from the pool of randomly selected schools. Altogether 332 eight-grade students were involved in the investigation, with an average age of 14.2 years. Students were videotaped during chemistry lessons, and their worksheets were collected afterward. The 12 chemistry teachers, who conducted lessons in these schools, were interviewed before the lessons; their teaching plans were also collected. The collected data was analyzed using qualitative methods. The results indicate that many teachers in Slovenian primary schools are not fully aware of the potential of experimental work integrated into chemistry lessons for the development of students’ experimental competence. Further research of the value of different kinds of training to support teachers for the use of experimental work in chemistry teaching is needed.


Introduction
2][3][4][5] Experimental work is essential in the teaching and learning of chemistry as it brings together several activities with different objectives, but its implementation is also a cost burden for the school.Therefore, the planning of its implementation needs to be based upon the effectiveness of the envisaged performance for achieving the learning objectives. 6,7This article focuses on a study of the integration of experimental work in the teaching and learning of chemistry in Slovenia from the perspective of the chemistry teachers' awareness of the potential of specific experiments integrated into chemistry lesson for the development of students' experimental competence, as well as teachers' ability to set learning objectives relevant to the teaching plan and use of learning materials related to experimental work in the classroom.

The Role of Experimental Work in the Teaching and Learning of Chemistry
Experimental work is generally considered the main method of teaching in science education; 3 therefore, it is an important building block of science education. 8,9Unfortunately, the teachers often use the experimental work in teaching chemistry mainly because it is required in the (national) chemistry curriculum. 10,11Hofstein, Kipnis, and Abrahams 4 indicate that teachers sometimes interpret the purposes and aims for learning with experimental work differently than specified in the chemistry curriculum.Because experimental work combines experimental activities with different objectives, 6 in the planning of such work, it is necessary to think about the effectiveness of students' foreseen activities for achieving the learning objectives. 7ne of the main objectives of the experimental work is that students recognise a connection between observa-Logar et al.: Effectiveness of Student Learning during ... tion and thinking about the observed phenomena, i.e. between the real world and the thought depicting the world.Tobin 12 stated that learning with experimental work is possible when the students can manipulate equipment and materials, and thereby simultaneously they build their knowledge of chemical concepts and related science content.Students should take advantage of experimental work to more easily understand the link between theory and the experimental activity. 13nfortunately, teachers often do not think about experimental activities as a primary main asset, which provides pupils with a sensible knowledge about science.Also many teachers do not involve pupils in the experimental work in a way that would encourage the development of science concepts and do not believe that they should help pupils to develop understanding between observation and science facts. 14Abrahams and Millar 7 found in the implementation of the experimental work that teachers are often aware only of learning of the new knowledge of chemical concepts, but not the purpose of the use of experimental work to develop understanding of scientific knowledge in general as well as the development of experimental skills.

2. Effectiveness of School Experimental Work
Millar et al. 6 proposed a four-step model for measuring the effectiveness of experimental work.
The starting point of this model is the teacher's learning objectives (A) or what he/she wants students to learn.This can be a specific part of the substantial subject knowledge or a specific viewpoint on the process of natural science research (e.g.collection, analysis, or interpretation of empirical evidence).Once a teacher decides on the learning objectives, the next step is the design or the selection of experimental tasks for the learners (B) to enable their achievement.The next stage of the model includes consideration about what students actually do (C) when they pursue the task.The last stage deals with what the students actually learn (D) during the experimental task. 6n this way, Millar's model distinguishes two meanings of effectiveness:  6 The important purpose of experimental work in science at school is to help students to make connections between real objects, materials, and events, and the abstract world of thoughts and ideas. 6According to Tiberghien 15 experimental work intended to help students is defined as a connection between the two levels of knowledge: the level of objects and observations (o) and the level of ideas (i).Abrahams and Millar 7 found that experimental work is appropriate for students to learn how to deal with laboratory equipment but is less efficient in facilitating students' learning based on the collected data for the development of scientific ideas.
Properly integrated experiments in the interpretation of the new concepts facilitate the linking and understanding of three basic levels of the perception of chemical concepts: macroscopic, submicroscopic or particulate, and the symbolic level. 16Therefore, the experiment (conducted either as a demonstration or an individual or group work of students) needs to be an integral part of any interpretation of chemical content in the class. 16,17To support students in meaningfully connecting the results of the experimental work with the discussed chemical concepts and in learning how to present them on the symbolic and particulate level, the teachers should strive for the integration of knowledge and experimental work. 18Furthermore, Solomon 19 stated that the observation of changes during the experimental work alone does not ensure students' understanding of presented concepts.Solomon 19 believes that learning with experimental work is effective if students' thinking is facilitated by connecting the visual perception of phenomena with the already known science concepts.
Mancy and Reid 20 found that experimental work can cause an overload of information for students, accompanied by very little the actual learning (in the sense of understanding).Johnstone and Wham 21 associated students' overload during the experimental work with the need to remember theoretical facts, names of apparatus and materials, written and oral instructions about the procedure and new skills.It is, therefore, reasonable that teachers plan experimental work in such a way that students focus on what is truly important, that they enable students to become familiar with the objectives of the experimental tasks prior to the experimental work, and that they provide students' the opportunity to discuss steps of the experimental tasks. 22n an attempt to reduce the cognitive load of learners and simplify the experimental work, teachers began to use the written instructions according to the step-by-step principle. 23Furthermore, in Hofstein and Lunetta's 14 opinion, the role of learning tools, such as instructions (workbook or worksheet) is important in the teaching of experimental work.In their opinion, teaching materials help learners in focusing attention.Well-prepared instructions provide information about exactly how something should be done and what students need to observe, which measurements must be carried out, and what information they need to collect in order to be able to answer the questions and form the conclusions of the experimental tasks.
To recall the most important information before the experimental work, it is sometimes suggested to use preexperimental activity, 24 which is perceived both as a means for reducing the workload with the information and also to interest the students in experimental work.The primary purpose of the pre-experimental activity is to focus and prepare students for learning. 24Reid and Shah 24 define the importance of the pre-experimental activity as encouragement for students to think during the experimental work; preparation of students for the experimental work; the guidance of students during their consideration of the procedures or chemical concepts; encouragement of students to connect the experimental work with the new concepts and previous knowledge, etc. Investigation of the effectiveness of pre-experimental activity confirmed its effectiveness and indicated that the students also form a more positive attitude towards experimental work in this way. 25brahams and Millar 7 propose discussing the experimental activities also in the follow-up lesson, which is defined as a post-experimental activity.Reid and Shah 24 propose that in the context of the post-experimental activity it is necessary to facilitate meaningful reflection on the experimental work, which can result in many benefits for the development of chemical concepts and processes.However, Abrahams and Millar 7 note that in the case that there is a pause between the experimental activity and the discussion about the experimental work, the efficiency of the understanding of tasks decreases, so it is best to carry it out directly after the completion of the experimental work.

The Context and the Purpose of the Study
Experimental work has a central role in chemistry curricula at all educational levels; 1-5 consequently, the planning and implementation of experimental work in the teaching process need to facilitate achieving the stated learning objectives.It was found that teachers sometimes interpret the purposes and aims for learning with experimental work differently than specified in the chemistry curriculum. 4With regard to experimental work Slovenian chemistry curriculum 2 indicates the objectives that are related to: (1) students' acquiring of content knowledge, (2) development of students' experimental skills and abilities, and broader (3) students' development of natural science competences.Because experimental work combines experimental activities with different objectives, it is necessary to think about the effectiveness of students' foreseen activities for achieving the learning objectives in its planning, as well the use of written materials. 6,7n the present article we use the term demonstrationbased experimental work when experimental activity is based on a frontal presentation of the experiment by the teacher or the individual student, while the remaining students observe its implementation.We use the term stu-dents´ hands-on experimental work to denote experimental activities in which students conduct all experimental activities by themselves.In order to address the holistic potential of experimental work in supporting students' de-velopment with regard to curriculum objectives, 2 the term students' experimental competencies is used.
The case study presented in the paper is part of a major multiple-case study, 26 aimed at examining teachers' concepts of learning and teaching.In particular, we focused on studying the authentic information from the school practice in Slovenia related to the implementation of experimental work in regular chemistry lessons at the primary level of education.Data collection, analysis and interpretations have been carried out within an interpretative constructivist approach. 27ith regard to the research aim, the following research questions (RQ) were defined: -RQ1: Are the chemistry teachers in primary schools aware of the potential of specific experiments integrated into chemistry lesson for the development of students' experimental competences?-RQ2: How are the worksheets used during the experimental work in chemistry lessons?-RQ3: Is teachers' awareness of the objectives of experimental work related to students' understanding of the experimental work?

1. Sample
In September-December 2011 we invited 52 schools, based on a stratified random selection from 452 primary schools across 12 statistical regions in Slovenia (list of schools available at the homepage of Ministry of Education, Science and Sport 28 ), to collaborate in the research.Fifteen schools from different statistical regions gave a positive response to the invitation.After discussing and coordinating the terms of the research with teachers, 12 schools from five statistical regions (Osrednjeslovenska, Gorenjska, Obalno-kra{ka, Notranjsko-kra{ka, and Jugovzhodna) decided to collaborate in the research.The research collaboration included 12 chemistry teachers and a total of 332 eight-grade students, with an average age of 14.2 years; 191 students took part in lessons involving students' hands-on experimental work, while the remaining 141 students took part in lessons involving demonstrationbased experimental work.To ensure anonymity, teachers' and students' data was collected by the use of code-names.In this article, the collaborating teachers are code-named by using the first 19 letters of the English alphabet from teacher A to teacher T (the letter Q is skipped), based on the nineteen recorded lessons.

2. Instruments
The following instruments were used for gathering the data in the interviews, which were audio recorded: derstanding of the usefulness of the obtained knowledge in everyday life.In each of the sections, students were asked three questions.The questions for students varied from one unit-lesson to another as different contents were used, in particular regular unit-lessons at their school.
During video recording of the unit-lessons in chemistry classrooms four cameras were used simultaneously, thereby one camera has been placed in each of the corners of the classroom.
The full texts of the instruments can be obtained by request from the authors.

Data Collection
In this study, we used multiple sources of data collection (individual interviews, classroom observation, and other documents) because: (a) each source provided data from a particular point of view; (b) the sources generated questions and suggestions for each other; (c) the various sources gave a variety of perspectives on particular beliefs and positions; and (d) the information obtained through one source could be cross-checked with the others. 29The research proceeded through several phases; an outline of the main phases of the investigation is presented in Advance Organiser (Scheme 1).
Before starting the research, the consent of the Commission for Ethics, Faculty of Arts, University of Ljubljana was obtained.Students', teachers', and parents' consent of free and conscious decision for the participation in the survey was obtained from all participants.
We video-recorded the unit lessons at Slovenian schools during the spring of 2012.The recording took place during regular chemistry lessons, and four cameras were used simultaneously.We recorded and observed ten unit-lessons involving students' hands-on experimental work and nine lessons involving demonstration-based experimental work for a total of nineteen lessons, comprised of twenty-one classroom hours.In addition to the video recordings of 19 unit lessons, we also collected the teac-hers' written lesson plans for the experimental work and audio-recorded interviews with teachers before conducting the unit lesson.We recorded unit lessons which lasted for one or two classroom hours, depending on the decisions of the collaborating teachers.After the recording, we randomly selected four students from each class, who were invited to take part in the interview (altogether 73 students were interviewed out of 332 participating eightgrade students -see details in Scheme 1).The interview was conducted without the presence of the teacher (only the student and the 1 st author were present).We also photocopied the learning materials (the worksheet, workbook, and notebook), which these students used during the unit lesson.
The teachers could address different contents in their lessons and experiments, as the specific learning contents of the experimental work were not the object of the study.In the studied lessons, the following contents of the curriculum for chemistry in primary school 2 were covered: (1) particulate nature of matter (N = 4), chemical reactions (N = 6) and acids, bases and salts (N = 9).

4. Data Analysis
The data collected was analysed in order to answer to the particular research questions as indicated in the Table 1.
Ten percent of particular data sources were independently analysed by two researchers (1 st author, 3 rd author) to ensure coding reliability and validity of the data. 30,31heir consistency in determining the categories was 91 to 95 percent.The inconsistencies between researchers were simultaneously synchronised through discussion.Eventually, the coding schemes were applied to all data.
In the analysis of the videotapes of the chemistry lesson were examined in time sequences of five seconds.For the purpose of this study, the Flanders Interaction Analysis Chart (Slovenian version by Marenti~-Po`arnik) 32 was elaborated for observations of the students and To determine the effectiveness of particular lessons, consequently various sources were analysed with the scheme for planning and evaluating of experimental work modelled after Millar et al. 6 on the basis of the analytical framework for evaluating of the efficiency of experimental tasks 7 .The theoretical framework of Millar's model 6 is in detail explained in the introductory part of the paper, accordingly the analysis was based on four steps (A -the teacher's learning objectives, B -the design or the selection of experimental tasks for the learners, C -consideration about what students actually do, D -what the students actually learn), that eventually determine the effectiveness of experimental work on level 1 (comparison of steps B and C) and the effectiveness of experimental work on level 2 (comparison of steps A and D).The rubrics developed for the purpose of the analysis can be found in the Appendices.

1. Chemistry Teachers' Awareness of the Potential of Specific Experiments Integrated Into Chemistry Lessons for the Development of Students' Experimental Competences
Based on the interviews conducted with chemistry teachers, we determined that few teachers (N = 3/19) in Slovenian schools are aware of all the foreseen curriculum objectives 2 they could achieve using experimental work, i.e. objectives related to students' acquiring of content knowledge, development of students' experimental skills and abilities, and students' development of natural science competences.
Similarly, to what Abrahams and Millar 7 found, also in Slovenia most teachers (N = 9/19) are aware of content objectives while planning the experimental work.
A typical teacher comment (Teacher D): "During experimental work students learn to understand, where some substances dissolve and where they don't dissolve, they learn why substances dissolve based on their structure."Some teachers (N = 6/19) are in addition to content objectives also aware of the objectives related to the deve-lopment of students' experimental skills and abilities.
A typical teacher comment (Teacher K):

"When students use hands-on experimental work, they learn to measure the temperature changes in chemical reactions and also how to correlate the changes of temperature with energy changes in chemical reactions."
Only a few teachers (N = 3/19) are aware of content objectives, objectives related to the development of students' experimental skills and abilities, and objectives related to broader nature science competence, as foreseen in the curriculum. 2 A typical teacher comment (Teacher R): "I believe that the purpose of experimental work is to enable the development of students in many ways, e.g.students should learn to establish a hypothesis; create a plan for the experiment; be able to recognize and write down the utensils and chemicals needed to carry out the experiment; designate the constants and variables; carry out the experiment; observe the changes during the experiment and write down their observations.They should also develop their experimental skills and abilities, take care of safety, analyse the results, and through all of this learn new chemistry concepts and understand the processes." Based on the results, it seems that many teachers in Slovenian primary schools plan the integration of experimental work into lessons primarily intuitively and that many teachers are not fully aware of the potential of specific experiments integrated into chemistry lesson for the development of students' experimental competences, which agrees with the findings of other researchers. 4, 7

2. The Use of Worksheets for Students During the Experimental Work
From the interviews with chemistry teachers, we found that for students' hands-on experimental work teachers prepare worksheets (N = 19/19), however, while planning demonstration-based experimental work the majority develops worksheets (N = 17/19).When comparing the results of the interviews with chemistry teachers and the analysis of the students' worksheets used during experimental work, we found that only a few teachers (N = 3/19) prepare worksheets for experimental work in accordance with their stated unit-lesson objectives in the interview.
Based on the analysis of video recordings from the chemistry lessons including experimental work and the accompanying worksheets, we see that students working with demonstration-based experimental work (Table 2) solve their worksheets better and with fewer mistakes in comparison with students doing students' hands-on experimental work (Table 3).Students taking part in demonstration-based experimental work finished 75.0-100.0%(Table 2) of all tasks on their worksheets, whereas students taking part in students' hands-on experimental work finished 18.8-100.0%of their worksheets (Table 3).Additionally, the number of correctly solved worksheets of the students taking part in demonstration-based experimental work is 94.8% (Table 2), while for students taking part in students' hands-on experimental work, only 81.0% of them solved it entirely correctly (Table 3).Therefore, the results show an advantage of the demonstration-based experimental work, which is based on the teacher having an active role, as he leads and directs the students towards observing and efficiently recording their results. 4espite the majority of teachers preparing worksheets for the students when using demonstration-based experimental work, they rarely give their students feedback after the work is finished (N = 2/7); when that happens, the tasks are solved with an accuracy of 95.7-100.0%(M = 97.9%).When the teachers do not check the worksheets, these are solved with an accuracy of 71.8-100.0%(M = 93.5%)(Table 2).
All teachers prepare worksheets for the students when using students' hands-on experimental work in chemistry teaching (N = 10/10).The high percentage of correctly solved worked sheets is seen when the teacher gives students feedback about the answers on the worksheet after the work is finished.In these cases, the work percentage of worksheets solved correctly varies 75.8-100.0%(M = 89.5%).When the teacher does not give students feedback after the work is finished, the percentage drops down to a range of 43.9-95.1% (M = 75.3%)(Table 3).
On the basis of the results, we can conclude that the students follow the instructions of the teacher well and accomplish the predicted assignments required to be followed during experimental work.However, all teachers are not aware of all the objectives of experimental work (as found in RQ1); therefore, they do not include these objectives into their worksheets, and, in these cases, the students also do not achieve all the objectives that they could.

3. Relation Between Teachers' Awareness of the Objectives of Experimental Work and the Students Understanding of the Experimental Work
Based on the analysis of interviews with the teachers and students in accordance with the scheme modelled after Millar et al., 6 we can summarise that the teachers' awareness of the objectives of experimental work affects the students understanding of the experimental work (effectiveness of experimental work level 2).This is the most noticeable for the teachers who are fully aware of all the objectives of experimental work as stated in the curriculum. 2 When planning demonstration-based experimental work (Table 4), we found out the teacher (N = 1/9) who is fully aware of all the objectives of experimental work has students who understand experimental work in 100.0% of their cases have appropriate skills and abilities in 89% of their cases.We found that teachers (N = 8/9) who are not fully aware of all the objectives of experimental work have students who understand experimental work in 0.0-50.0%(M = 29.6%) of the cases and have appropriate experimental skills and abilities in 16.7-70.8% of the cases (M = 47.7%).
Regarding students' hands-on experimental work, we can see in Table 5 that teachers (N = 2/10) who are fully aware of all of the objectives of experimental work have students, who understand experimental work in 50.0-75.0%(M = 62.5%) of the cases and have appropriate experimental skills and abilities in 62.5-79.2%(M = 70.9%) of the cases.However, when planning students' hands-on experimental work, we found that teachers (N = 8/10) who are not fully aware of all the objectives of experimental work have students who understand experimental work in 12.5-75.0%(M = 31.3%) of the cases and have appropriate experimental skills and abilities in 50-66.7%(M = 59.1%) of the cases.
It can be concluded that the teacher's holistic awareness of the objectives of experimental work is connected with the knowledge of the students both in demonstrationbased experimental work and in students' hands-on experimental work.Therefore, it is essential that during experimental work teachers help their students understand the connection between the practical activity, which represents the macro level and the theory, which represents the particulate/symbolic level.This causes the students many problems, as it is difficult for them to connect their observations and nature science concepts and apply them to experimental data and the experiment's conclusion.

Conclusions and Implications for School Practice
The research has indicated that many teachers in Slovenian primary schools plan the integration of experimental work into lessons primarily intuitively and that many teachers are not fully aware of the potential of specific experiments integrated into chemistry lessons for the development of students' experimental competences, which is a novel research finding for Slovenian school practice, but has been found also in other countries. 4,7,33It was found, that in Slovenia about a half chemistry teac- hers are aware of the content objectives while planning the experimental work, but only about one third is also aware of the objectives related to the development of students' experimental skills and abilities.Surprisingly, only few teachers (about one sixth of the teachers in our sample) is aware also of the objectives related to broader nature science competence, as foreseen in the chemistry curriculum. 2 Additionally, in defining objectives of experimental work, it is sensible to keep in mind the potential of using experimental work to motivate students; 4,8,13,16,[35][36][37] to popularize the natural sciences; 4,13,38 and to facilitate students' ability to communicate. 13,35,38When selecting the content of the experimental work in accordance with the objectives of experimental work, the teachers should strive for the tasks to be based on either the students' experiences or everyday life 38,[40][41][42][43] and select and appropriate form of experimental work based on its objectives (demonstration-based experimental work/students' hands-on experimental work). 4,13,24,46 novel finding for Slovenian school environment derived from the presented research is also, that the majority of students can follow the instructions of the chemistry teacher well and accomplish the predicted assignments required to be followed during experimental work (effectiveness level 1).Therefore, it is essential that teachers understand how important it is, that they develop adequate written instructions for students' chemical experimental work both in demonstration-based work and in students' hands-on work.Namely, it was found, that in ca-ses, where the teachers are not aware of all the possible objectives of experimental work and consequently that they neglect including these objectives into their worksheets, the students also cannot achieve all of the possible objectives that they could (effectiveness level 2).][51] For the improvement of the quality of experimental work in Slovenian primary school practice, it seems important, that in the future more attention is devoted to the development of pre-service and in-service teachers' understanding of the holistic perspective of the objectives for particular experimental work integrated into chemistry lesson.Although in last decades substantial efforts have been devoted to chemistry teachers' development of an understanding about students' development through implementation of the curriculum framework 1,2,34,47 and the use of experimental work in chemistry teaching, 16,17,48,[52][53] further research of the value of different

( 1 )
effectiveness level 1 -what the teacher wanted the students to do and what students were actually able to do (comparison of steps B and C), and (2) effectiveness level 2 -what the teacher wanted students to learn and what students actually learned (comparison of steps A and D).

Scheme 1 :
Scheme 1: Research design of the investigation

Table 1 :
Research questions and data sources analysed Logar et al.: Effectiveness of Student Learning during ...

Table 2 :
Analysis of collected data when evaluating the worksheets, the students used during demonstration-based experimental work.

solving worksheets while taking part in demonstration-based experimental work
*Percentage of correctly solved worksheet among solved worksheets.

Table 3 :
Analysis of collected data when evaluating the worksheets, the students used during students' hands-on experimental work.

solving worksheets while taking part in demonstration-based experimental work
*Percentage of correctly solved worksheets among all the solved worksheets.

Table 4 :
Analysis of collected data when evaluating the knowledge of students when the teacher is/is not aware of all the objectives of demonstration-based experimental work.

Table 5 :
Analysis of collected data when evaluating the knowledge of students, when the teacher is/is not aware of all the objectives of students' hands-on experimental work.

student answers in the interview after Teachers' answers in the interview before the unit lesson about: the unit lesson about the objectives
of training to support pre-and in-service teachers in their development for the use of experimental work in chemistry teaching is necessary.
Logar et al.: Effectiveness of Student Learning during ... kinds