DIFFERENT USE OF E-PORTFOLIO: EVALUATION & INTERPRETATION

1. Methodology

Interviews were conducted and analysed using MAXQDA as part of the research question of what similarities and differences can be identified in the perspectives of different target groups (students, teachers, pupils) with regard to the use of ePortfolios. The analysis was carried out both by coding and categorisation according to Mayring and by inductive category formation, whereby the categories emerged from the material. In addition, an analysis according to Mayring was conducted in order to gain deeper insights into the results.

To ensure systematic data collection, a catalogue of questions was drawn up in advance in close cooperation with various stakeholders, including the management of the Centre for Teacher Education (ZfL) at Justus Liebig University Giessen, consisting of Martin Reinert and Jeanne Flaum, as well as Claudia Spieler, the student group's internship officer, and Kristin Karl, an Erasmus project employee with extensive knowledge of ePortfolios and Mahara.

The survey is aimed at different target groups. These include students from Justus Liebig University Giessen, who are completing their general school internship (ASP) at the Max-Beckmann-Schule (Max Beckmann School) in Frankfurt. The Max-Beckmann-Schule is an upper secondary school where ePortfolio concepts are applied in different ways in the classroom. In addition to the students, teachers and pupils from this school are also interviewed. The interviews were conducted in five pupil groups, whereby the number of participants in the groups varied.

1. Methodology

In order to analyse the interview, the recordings were transcribed using a transcription AI. The transcripts serve as the basis for the analysis.

Qualitative content analysis is a common method of analysing qualitative interviews. The most frequently used method for interviews is structuring content analysis, in which a coding guide (e.g. according to Mayring or Kuckartz) is created. In order to create a coding guide, the statements from the interview must be assigned to different categories. Categories are deductively derived from the available data material (Mayring 2008, 602f). 

Table 1 Example of coding according to Mayring

By using this procedure, based on the questionnaire, a total of 6 categories could be defined. As this is a pre-structured interview, this coding can also be applied to the question catalogue. No further categories emerged in the course of the interview. The following is an overview of the classification of the question catalogue based on this coding: 

Table 2 Categorisation of the interview questions

2. Evaluation

The interviews are analysed below with regard to the categories. The three different target groups should be related to each other. The citation refers to the analysis scripts in MAXQDA. Due to the anonymisation of the participants, only a few sample excerpts from the transcripts are made available and personal data is made unrecognisable.

The aim of analysing the interviews is to gain an understanding of the individual needs and perspectives of the different target groups. On this basis, targeted measures can be developed to enable a sustainable learning and teaching culture when using ePortfolios in lessons, studies or other learning and teaching contexts.

The abbreviations of the interviewees are as follows:

• Students (S1-3)
• Teachers (L1-4)
• Pupils (SuS1-5)

2.1. Use

“Use" refers to how the individual respondents use ePortfolios. The focus is not only on how the interviewees use ePortfolios during the ASP, but also in which contexts they have already used ePortfolios and corresponding tools.

In the “Use" category, the following codes were defined according to Mayring:

• First time use
• Subject-related statements
• Purpose of use (retrospective, self-reflection, teaching/study-related);
• Purpose of use (teaching/study-related), more precisely:
• Accompanying
• Phases/projects
• Internship

1. Students

The three students interviewed worked with Mahara for the first time as part of the ASP. S1 and S3 generally worked with ePortfolios/the portfolio concept for the first time, while S2 had already created ePortfolios with the Pages tool. S3 says that during the internship he was not able to create and submit the internship portfolio with Mahara, but filled a USB stick with media content.

2. Teachers

The four teachers each use Mahara in different contexts.

L1 and L3 tended to use ePortfolios in the context of preparing for the A-Level, i.e. in a specific phase, but sometimes also during lessons. L4, on the other hand, shows intensive use of Mahara in relation to projects/phases, especially in preparation for the A-Level, as well as during lessons. L2 says that she has not yet been able to use it at the school, but that she has already gained experience at her training school in the past.

3. Pupils

The portfolio concept is an integral part of teaching in some subjects at the Max-Beckmann-Schule Frankfurt. All pupil groups have similar terms of use. In some subjects, ePortfolio concepts are used more intensively than in others. They also use it in project phases, to prepare for the A-level, for homework and occasionally for self-reflection (SuS4)

2.2. Understanding

The "Understanding" category refers to the respective conceptualisation of "ePortfolio", i.e. the electronic portfolio.

Using Mayring's coding guide, the following codes were identified in the "Understanding" category:

• Digital, online
• Multimedia
• Collection
• Tool-related statements (e.g. Mahara)
• Comparisons with the analogue portfolio
• Different portfolio types

1. Students:

Despite different levels of experience with the ePortfolio (see Use), all three students have a similar conceptual understanding of ePortfolios, which includes the following aspects, which were interpreted as coding according to Mayring: ePortfolios are a collection of different artefacts of different media content that can be presented digitally and/or online. For example, S1.

2. Teachers:

The teachers' definitions generally do not differ from those of the students. However, they supplement the definition by defining ePortfolio types and their purpose more precisely and in a more pupil-centred way. L1 and L4 describe the ePortfolio in a very competence-centred way and thus as a competency portfolio. L2 also mentions the development portfolio as an ePortfolio type.

3. Pupils:

The definitions of the pupil groups also hardly differ in comparison with the teachers and students. However, SuS4 and SuS5 also mention the portfolio tool Mahara as part of their definitions.

2.3. Added value

What added value do the respective groups see in relation to the portfolio work? The following codes could be defined for this purpose:

• Data protection, copyrights
• Digital competences
• Digitalisation in general
• Subject-related
• Individual needs in terms of self-presentation
• Communication, public
• Control, feedback
• Motivation, emotions
• Multimediality
• Learning type
• Retrospective
• Organisation, planning, collection
• Self-reflection
• Tool-related

Students:

The students see the added value primarily in the fact that a collection of learning products is created that can be accessed at a later date. These could be called up quickly (S2) and ideally you would have an overview of the entire course content (S1). The fact that multimedia content can be embedded in ePortfolio tools such as Mahara means that there is great scope for creativity and individualisation. This allows you to align yourself to your own learning type. Furthermore, this enables reflection processes (self-reflection as well as reflection with mentors), among other things, because content can be cross-linked and can be retrieved at any time, but can also be compared with other people. This means that "peer-to-peer control" (S3) is also possible. S1 and S2 also see possibilities in Mahara and the ePortfolio concept as a teaching tool/method.

Teachers:

Teachers see a wide range of added value in the use of ePortfolios in the classroom. They emphasise flexibility, multimedia and application possibilities. L1 emphasises getting to know the pupils better for diagnostics and monitoring as well as the possibility of presenting competence development and individual learning processes. The use of ePortfolios promotes open lessons and dialogue. L2 and L3 emphasise support in diagnostics, the promotion of personal responsibility and interdisciplinary skills as well as differentiation and self-directed learning. L4, who uses ePortfolios most intensively, emphasises the advantages of flexible working, accessibility and the promotion of digital skills. 

Pupils:

The pupil groups see various advantages in ePortfolios. Group SuS1 emphasises the versatility of Mahara for presentations, group work and task planning. For SuS2, the focus is on multimedia and practical application, whereby familiarisation with the tool was motivating. SuS3 sees advantages in the visualisation of scientific content and time savings in preparing for the A-Level. Group SuS4 uses ePortfolios to improve grades and promote digital skills. SuS5 appreciates the support in lessons, the versatility and the creative freedom. Overall, all groups see ePortfolios as an important resource that can be used individually and creatively. They say that certain subjects are better suited to ePortfolio work, such as SuS3 in science subjects and SuS2 in language lessons.

2.4. Challenges

The participants were asked about the individual challenges they faced when working with ePortfolios. This results in the following codes:

Application-related (in relation to the use in the respective teaching/learning scenario)

• Subject-related
• Control, feedback
• Learning type
• Extra work
• Motivation, emotions
• Technology, tool-related
• Clarity

Students:

All students see the fact that working with ePortfolios involves a comparatively high amount of extra work as a challenge. The "cost-benefit ratio" (S1) would be imbalanced if it had to be managed throughout the degree programme, but the familiarisation with the tools and the development of digital skills relevant to ePortfolio management would also be relatively high, as the tools are complex and have their own technical peculiarities and problems (S2). This means that some of the tools are not accessible enough from the student teachers' perspective, even if they were to use them for their own lessons. Furthermore, some subjects are rather unsuitable for ePortfolio management (S1).

Teachers:

L3 and L4 agree with the students that the tools are more difficult to access and that technical problems can occur. L2 mentions the extra work involved, while L1 emphasises that not all subjects are suitable for use.

L4, who has the most experience with ePortfolios, sees the greatest challenges and mentions the most points in her survey. She emphasises the balance between coaching and pupil observation as crucial and criticises educational policy problems and the discrepancy between traditional forms of assessment and ePortfolios. She also mentions colleagues who are sceptical about the concept and thus find it rather difficult to implement it uniformly at a school, as well as the adaptation of learners to new requirements as challenges.

Pupils:

The pupils see similar challenges in dealing with ePortfolios. Everyone finds Mahara cumbersome to use, especially when creating texts, and criticises the old-fashioned design and problems with the display on different devices. In addition to the high expenditure of time, SuS1 and SuS2 sometimes find the use of ePortfolios unnecessary and fear a loss of overview with too many ePortfolios. They feel that their work is not adequately recognised by teachers. SuS3 also criticises the difficulty of getting started and the lack of support from the teachers. SuS4 feel controlled by the teachers. Some pupils prefer analogue writing and doubt the usefulness of ePortfolios for their learning process. In general, all groups express feelings of frustration, monotony, excessive demands and effort in relation to working with ePortfolios, especially during the assessment process.

2.5. Changes/effect

The category "change/effect" refers to what has specifically changed in relation to teaching. The following codes were identified:

• Digitisation
• Effort (in the neutral sense)
• Organisation

Students:

As the students had little or no previous experience with ePortfolios and this type of work had not yet been carried out (see Use), they were not asked any questions on this topic.

Teachers:

Teachers offer insights into various aspects of the changes in their own teaching and the pupil experience that they themselves perceive. As teachers use ePortfolios in different ways (see Use), there are various aspects that need to be analysed.

L1 emphasises that the teaching materials are constantly accessible to both pupils and teachers and that this can enable a structured design of lessons and teaching materials. In addition, a feedback culture among the pupils is promoted, which serves to learn from each other, whereby more personal responsibility also lies with the pupils themselves. L2 also sees a change in the assumption of personal responsibility, whereby the teacher takes on more of an observer role. She also points out that the preparation and evaluation of lessons represents a considerable amount of extra work. Compared to conventional lessons, L3 gives pupils more freedom and invests time to address individual problems. However, she also recognises that evaluating lessons is time-consuming and requires a lot of consultation and insight. However, the pupils would be more creative and intrinsically motivated, even beyond the requirements of the work assignment. L4 maintains open communication with pupils and promotes a stronger feedback culture. As a result, she develops a better understanding of them and promotes a co-operative approach that is more akin to coaching, which is in line with the aforementioned observer role. At the same time, she warns of possible stressful situations, especially if pupils do not like the ePortfolio concept and feel that they are being watched.

Pupils:

All groups state that they have noticed extra work. They also state that they are experiencing increased digitalisation in the classroom, which has an impact on their working methods and their interaction with the materials, as well as on their skills development in dealing with digital media. In this context, SuS2 emphasises that they have to pay more attention to data protection and copyrights than in conventional lessons.

2.6. Wishes

What would the respective respondents wish for when implementing the ePortfolio concept in their studies or lessons? The following codes were identified during the analysis:

• Education, support, assistance
• Education system
• Tool- and technology-related
• Self-determination
• Standardisation
• Appreciation

Students:

In order to be able to implement the ePortfolio concept profitably, the students would like to see changes in its use at the university. A clear order and structure as well as guidelines for the standardisation of ePortfolio work should be specified (S1, S3). S1 states that the structure and time-intensive nature of the teacher training programme does not tend to favour ePortfolio work. This means that it is currently rather difficult to establish links to the further phases of teacher training. 

In addition, the visibility and thus the accessibility for students should be increased; this could be ensured by organising more informal events and embedding ePortfolio work in different seminars (S2). One of the students also expressed wishes for improvements to the tool, such as the registration process, correction functions or layouts and template use (S3).

Teachers: 

L1, L2 and L4 express the wish for more acceptance of the ePortfolio tool Mahara and would like more colleagues to engage with the topic of ePortfolios and recognise the added value, as well as for there to be more coordination between them when using them. L1 is also open for the integration of other media such as ChatGPT. L3 would even like to be allowed to use ePortfolios as an examination format in eleventh grade.

Pupils:

Almost all groups express a desire for standardisation of use. SuS1 favours the use of ePortfolios only for larger tasks and group work, but not for homework or notes. Group 2 calls for a clearer design of the ePortfolios and suggests limiting their use to certain subjects. With regard to the technical aspects, they would also like to see a simplification of administration via the school portal (SuS1). They would also like to be able to find people more easily on Mahara and for some functions in Mahara to be simplified (SuS2)^[1]. SuS3 emphasises the desire for appropriate appreciation for the extra work involved in using ePortfolios and calls for a better way to share content with the whole school. They would also like to harmonise the portfolio concept and its use with other teachers. SuS5 would like to have the freedom to decide for themselves which tasks they want to use ePortfolios for. For example, SuS4 suggests getting to know different methods, including analogue methods, at the beginning, and then being allowed to vote on whether they want to use ePortfolios.

3. Conclusion and literature

A brief summary will draw conclusions from the interviews with the three groups (students, teachers, pupils) regarding the use of ePortfolios.

It is evident that all groups identify significant challenges. In particular, the technical complexity of tools such as Mahara and the extra work involved are perceived as difficult. Assessing the benefits is also challenging.

Students and pupils who are not directly involved in assessment processes cite more challenges than teachers, especially L4, who works intensively with ePortfolios. Many of the challenges are seen in education policy and institutional education systems, which make the effective use of ePortfolios more difficult. Teachers report positive changes in the classroom, such as increased pupil responsibility and an improved feedback culture. Pupils are experiencing increasing digitalisation in the learning environment.

To summarise, the analysis shows that ePortfolios offer potential for promoting learning and development processes, but are associated with considerable challenges in existing educational structures. A targeted approach that takes into account the needs and perspectives of all those involved is crucial for the successful implementation of ePortfolios in an educational context.  

4. Literature

Mayring, Philipp (2008). Qualitative Inhaltsanalyse. Grundlagen und Techniken (10. Aufl.). Weinheim: Beltz.