Effectiveness of educational leadership through directors' performance in online higher education. The biggest online university in Spain

Introduction

The use of information and communication technologies (ICT) nowadays is standardized in teaching-learning processes. This reality implies that educational organizations must be transformed to become flexible and innovative structures, open to new teaching methods and didactic resources, putting at the center the individual capable of reflecting and analyzing their environment, highlighting the constructivist methodology in which the student builds his own knowledge through autonomous work processes and the use of different technological tools (Palomino and Ramos, 2021).

In this context, virtual education is a reality characterized by being a teaching-learning process based on the principles of an active pedagogy, in which the student is responsible for participating frequently and effectively in different virtual environments (Aurangzeb and Mazhar, 2020). Unsurprisingly Chwen-Li et al. (2022) state that we can consider virtual learning as a new paradigm in the learning process, as it does not require face-to-face interaction and can be done without relying on a traditional classroom environment. However, as Cordie and Lin (2018) state, the prevalence of virtual learning leads to unique challenges, including dealing with various stakeholders, providing an effective product to students and establishing new paradigms on how to provide educational components.

Likewise, a new element stands out as an important part of this process: the reality of leadership. Indeed, leadership has emerged not only as a fundamental element for institutional success in achieving this transformation but also as a key factor in improving student performance. Not surprisingly, leadership in education is becoming a major factor in the improvement of education and a political priority. Indeed, in Marichal-Guevara et al. (2018), we find how various studies conclude that the exercise of leadership in education determines, in an important way, the quality of education.

When it comes to improving the quality and results of students in higher education institutions, different leadership models have been proposed; however, in of all of them, it is worth highlighting the role of pedagogical leadership in improving both the performance of students and the quality of universities in an effective way.

That is why in higher education institutions that teach their studies in the online modality, we will refer to pedagogical e-leadership to refer to that model of pedagogical leadership that is developed in virtual environments, since this reality will also influence leadership in educational institutions, facilitating the student both to reflect and analyze their environment from a constructivist approach, in which he himself builds his knowledge through autonomous work processes (Ibarra et al., 2023).

Indeed, in virtual educational environments, the context of traditional leadership changes, since they allow us to develop and carry out a multitude of jobs, projects and tasks at any time and in any place, both in real and virtual spaces. In fact, some authors define e-leadership not only as an extension of traditional leadership but as a fundamental change in the way in which leaders and people relate to each other within organizations, as well as the way in which different organizations relate to each other (Tintoré and Gairín, 2022).

We cannot ignore the fact that e-leadership takes shape in a context where collaboration and interaction, which seek to create and distribute an organizational vision in which goals, organizations and individuals are united, will be mediated by ICT (Antonopoulou et al., 2021).

Along the same lines, Avolio and Kahai (2003) state that, although e-leadership is a concept that emerged relatively early and with continued conceptual ambiguity, there will be significant differences between leading a traditional organization and leading those that have virtual environments mediated by ICTs, since these environments seem to require leaders who can deal with the paradoxes and dilemmas, as well as the complexity of the behaviors associated with the use of ICTs. It is essential to consider this nuance since educational e-leadership is based on the whole theory of leadership, but it does so assuming the characteristics of its nature and content.

Miller and Ives (2020) state how e-learning has entered the mainstream of higher education as a strategic change agent, and this transformation requires e-learning leaders to develop the skills to successfully innovate at a time of increased competition and rapid technological change.

On the other hand, when we transfer the concept of leadership to the field of education, the concept of pedagogical leadership stands out, understood as that which is oriented toward improving student performance and results and is widely recognized as an important and effective element in improving educational quality. If, on the one hand, the first factor that has a significant impact on student learning is the teaching staff, the second element that can greatly influence the performance of a faculty, by facilitating a series of conditions and environments to improve the context in which it can better perform its work and students improve their learning, will be leadership (Anderson, 2010).

Not surprisingly, as Harris et al. (2022) assert, leadership in educational contexts can have a particularly positive influence on both institutional and student outcomes when it is distributed. Indeed, management and quality are related in many ways, and a strong leadership position enables distance education programs to achieve their goals successfully and efficiently while remaining responsive to the needs of their beneficiaries and users (Chang et al., 2022).

However, in a study conducted by Palomino et al. (2022a) in which they analyze the incidence of pedagogical e-leadership in higher education institutions from the most current scientific literature, we find how the scarce amount of work in this line indicates that research is still insufficient, thus evidencing the need to further deepen this area of research, which will revert in the efficiency and effectiveness of organizational development, as well as in the improvement of all dimensions of educational institutions of higher education. So, when it comes to understanding this reality, it is convenient to start from the fact that “Higher Education institutions are complex organizations in which their management is a challenge for leaders” (Smith, 2020, p. 39).

That is why the need arises to continue the research work observing the necessary scientific standards that ensure the quality of knowledge, to understand to what extent the influence of learning management in higher education, and specifically in online training contexts such as The International University of La Rioja (from now on UNIR), is aimed at increasing and improving the performance of its students, as well as in improving the quality of institutions. This, then, is the research problem from where we start this study.

UNIR was created with the intention of allowing access to higher education to all people who, for different reasons, cannot study in person and who need to combine their learning with the different tasks they do every day. To achieve this goal, they have developed a unique methodology, with a strategic plan that seeks to ensure quality education. The features that define this methodology are the following: effective teaching focused individually on each student, adapting the pace of their training to their specific circumstances – UNIR provides a tutor to each student, to make it easy to provide individual tuition, finally, UNIR ensures the full commitment of faculty and management staff to the success of each student (UNIR, 2022).

Among the values that UNIR marks as a university, the following stand out: an offer of degrees and programs that meet the needs of society; careful attention to the quality of teaching and resources; commitment to student success; personalization of teaching tasks with the accompaniment of the student throughout their learning process; ability to respond to the expectations of stakeholders; and constant attention to innovation in teaching and learning methodology (UNIR, 2022).

UNIR has 7 faculties and schools, in which 154 degrees and official masters and 95 own degrees are taught. The Faculty of Education currently offers 15 degrees, 23 official master's degrees, 7 own master's degrees and 17 continuing education degrees.

The UNIR team is made up of teaching and research staff (PDI) and management and administration staff (PGA). Both work in the same direction so that everything works effectively in each sector of the university.

Objectives

Methodology

The methodology used to achieve our objectives has been a descriptive and quantitative methodology through a cross-sectional study based on the implementation of the questionnaire: Adaptation of the VAL-ED to the university context, developed by Palomino et al. (2023).

These authors highlight how, for decades, educational research has focused on finding ways to improve leadership effectiveness, valuing managerial evaluations as an important tool that can detect those improvements. However, developing these assessment tools has always been challenging, because they vary greatly from each other in both content and focus, as well as different perspectives on what leadership behaviors should be assessed. In response to this need, in 2005, a team of researchers from Vanderbilt University and the University of Pennsylvania, both in the United States, developed a valid, reliable and theoretically grounded assessment model of managerial educational leadership that has recently been adapted to the context of higher education. Notably, the VAL-ED assumes that effective student-centered leadership lies at the intersection of two dimensions: the core components built through key processes.

As we find in Palomino et a. (2022b), it is important to note that the VAL-ED has been designed to evaluate critical leadership behaviors to perform diagnostic analysis, performance feedback, progress tracking, professional development planning and summative evaluation. These authors highlight how there have been different studies in which the validity and reliability of the VAL-ED have been tested:

Polikoff et al. (2009) evaluated the differential operation of VAL-ED elements. Porter et al. (2011) studied the validity and reliability of VAL-ED through a nationwide study in the United States. Minor et al. (2014) also wanted to check the validity of VAL-ED in primary and secondary schools in the United States. They carried out a study in which they analyzed the accuracy with which the VAL-ED scores can identify the belonging to the two groups previously selected by the superintendents (directors whose performance of their functions was in the top 20% and the bottom 20%). Goldring et al. (2015) investigated the psychometric characteristics of the Vanderbilt Assessment of Leadership in Education, analyzing its convergent and divergent validity. They hypothesized that VAL-ED is highly correlated with other measures of instructional leadership. Likewise, in their study, Goff et al. (2015) highlight how numerous researchers have conducted several studies to validate VAL-ED, proving to be a reliable instrument that can be used in multiple contexts. Minor et al. (2017) evaluated the test-retest validity of the VAL-ED. They found that the principal and teacher grades of moment 1 and moment 2 have significant, positive and meaningful correlations. With these results, the Vanderbilt Evaluation of Leadership in Education (VAL-ED) instrument is, therefore, a reliable and valid instrument to measure the effectiveness of educational leadership.

On the other hand, we know that when assessment instruments are used as intended with a sufficiently significant sample for each of the groups that have interacted with a director, these instruments (in our case the adaptation of the VAL-ED in the context of higher education) will produce reliable and valid results on key leadership behaviors focused on learning, facilitating interpretations that will allow directors to discover areas for improvement that affect leadership focused on more effective learning (Palomino et al., 2022b).

Data collection tool

The questionnaire used to evaluate the pedagogical leadership of directors has been the translation and adaptation of the VAL-ED to the Spanish university context developed by Palomino et al. (2023). These authors considered theoretical and empirical antecedents, as well as the judgment of several experts in the subject of the universities of Granada and the International University of La Rioja to validate the adequacy of the questions in the university context. It was also validated by a graduate in translation and interpreting, a graduate in English philology and a graduate in Hispanic philology to guarantee the correct translation.

Since leadership focused on effective learning is at the intersection of two dimensions, the main components created through key processes, the VAL–ED has been designed to evaluate both the main components and the key processes. The main components refer to the characteristics of educational institutions that support student learning and improve the ability of the teaching staff to teach their students. On the other hand, key processes refer to how leaders create and manage those major components.

Sample

The sample has been obtained from the Faculty of Education of the International University of La Rioja. Of the 38 official degrees offered at the Faculty of Education of the International University of La Rioja, 11 participated in the study, obtaining the following sample of participants: 11 directors, being 3 degree directors and 8 directors of official master's degrees; 5 supervisors of the directors of the 11 participating degrees, with 2 supervisors of directors of bachelor's degrees and official master's degrees, and 3 supervisors of directors of official master's degrees; finally, a total of 89 professors from the 11 degrees participated in the study.

In this way, a participant sample formed by 105 people, representing 100% of the directors of the degrees; 100% of the supervisors of these, and more than 75% of the teachers who teach in each degree was obtained. We should take into account that neither the directors of the master's dissertation subject, nor the teachers of the practices subject, have participated. The sample represents in a heterogeneous way the different degrees of the Faculty of Education and the interested parts.

Procedure

To evaluate the leadership evidence of the three agents involved (faculty, directors/coordinators and supervisors), data collection was carried out between May 2021 and February 2022. In all, 105 questionnaires were collected. The instrument was applied in digital format, and the participants in the research were previously informed about the objective of the study. Their participation was voluntary, and responses confidential and anonymous.

To achieve the potential of VAL-ED for its intended purposes and to be managed as designed, as suggested by Vanderbilt University (2011a, b), the following aspects were considered in both the key usage summary and the implementation guide:

1. The teachers invited to complete the VAL-ED had to result in a representative and reasonably large sample. In this sense, we obtained 89 responses that, although they do not represent the 100% percentage of faculty members who should have responded to the surveys, we can consider as a sufficient number of responses to carry out the study. On the other hand, it should be noted that, since both the appropriate supervisors were included, together with the directors and the faculty members of the degrees, the result is a 360-degree evaluation.

2. Similarly, the evaluation was coordinated by a neutral and objective person who did not complete the VAL-ED.

3. It is recommended, on the other hand, that the evaluation should only be carried out, at the earliest, during the end of the second month of the academic year, as this increases the likelihood that respondents have had a reasonable opportunity to interact with the director they are evaluating. In this sense, it should be noted that the questionnaires were collected at the end of the second semester of the 2020–21 academic year and at the end of the first semester of the 2021–22 academic year.

4. Teachers were guaranteed that their answers were anonymous, and they were allowed adequate time to read, reflect on the evidence, and rate the director's behavior.

Analysis and interpretation of results

As we found in Vanderbilt University (2011a, c), the VAL-ED behavior inventory provides information on a total score, six subscales for core components, six subscales for key processes separately for each group of respondents and an overall average among the groups of respondents.

The most important score that results from the VAL-ED is the overall total effectiveness score of the director. This score is based on the average ratings of all respondents, where each group of respondents has the same weighting and is reported on the 5-point effectiveness metric used to rate each of the 72 elements of the instrument. Therefore, the director's overall total effectiveness score and core component and key process subscale scores are reported on a continuous scale from a minimum of 1.0 (non-effective) to a maximum of 5.0 (very effective). The various effectiveness subscale ratings for core component and key process behaviors fall within this same operating range.

Results

As discussed before, the VAL-ED behavior inventory provides information on a total score, six subscales for core components, six subscales for key processes separately for each group of respondents and an overall average among the groups of respondents. The core components and key processes are based on the same information, so while their information is redundant, the two separate profiles offer diagnostic information on how a director's behavior can be improved to achieve a more effective institution and, in turn, improve student performance.

The data obtained allow us to analyze the pedagogical e-leadership in the Faculty of Education of the UNIR, as well as identify the effectiveness of behavior in the leadership of the directors taking as reference the competence standards of VAL-ED, as well as verify which areas of leadership behavior represent the greatest strengths, limitations and main needs for improvement.

As can be seen from Tables 2 and 3, scores obtained at the intersection of the “high standards for student learning” core component with each of the key processes result in distinguished performance by directors in each of them. The scores obtained at the intersection of the “rigorous curriculum” core component with each of the key processes result in a distinguished performance in all these, except for the key component “planning”, in which the performance would be competent.

Regarding the scores obtained at the intersection of the main component “quality instruction” with each of the key processes, it is worth noting how we find a distinguished performance in all of these, apart from the key component “Monitoring”, in which the performance would be competent. In addition, scores obtained at the intersection of the main component “culture of learning and professional behavior” with each of the key processes result in distinguished performance by directors in all of them.

It is noteworthy, however, how the scores obtained at the intersection of the main component “Connections to external community” with each of the key processes result in competent performance in each of them, except for the key component “Supporting”, in which the performance would be basic. Finally, the scores obtained at the intersection of the main component “performance accountability” with each of the key processes again result in distinguished performance by directors in all of these.

When we compare the performance results obtained by each of the groups of respondents – directors, supervisors and faculty (Tables 4–6) – to determine to what extent the three groups of respondents (faculty, supervisors and directors) agree on the effective performance of the pedagogical e-leadership of the directors, as we proposed in the second specific objective, we observe how the results of the interaction of the main components “standards for learning of high students” and “rigorous curriculum”, coincide with all of them obtaining a distinguished assessment in the performance of the directors.

In the case of the main component “quality instruction”, the level of performance obtained in four of the key processes coincides, finding differences in the key processes of planning and monitoring in the case of the group of respondents of teachers; and advocating and monitoring in the group of respondents of supervisors, who rate them with a performance as competent, compared to the distinguished with which directors are evaluated in all of them.

When we analyze the results of the main component “culture of learning and professional behavior," the level of performance obtained in the six key processes coincides when we compare the responses of the groups of directors and teachers. Both groups qualified it with the level of distinction. However, when comparing the scores of the group of directors with that of supervisors, we find differences in the key process of monitoring: while directors rate performance with the level of distinguished, the group of supervisors qualifies it as competent.

It is in the results of the main component “connections to external community” that we find greater differences in the performance scores of each of the key processes. The first thing that stands out is that it is the component in which the lowest scores have been obtained in all groups of respondents. On the other hand, it highlights how it is the supervisors of the directors and the directors group of respondents that give the lowest scores to the performance of the key processes of this component, since the highest level of performance obtained is that of competent in the communicating process, having being valued with the lower than basic level. The teachers value a competent performance in four of the processes evaluated: implementing, supporting, communicating and monitoring, compared to the three that we find in the group of directors: implementing, supporting and advocating.

It should be noted that in the main component “responsibility for performance,” it is the group of teachers that give the lowest score to the planning process, qualifying it with a competent performance, compared to distinguished that the groups of directors and supervisors grant. In the rest of the processes of this component, all groups of respondents agree in their evaluation, evaluating the performance of directors with a distinguished level.

As we have discussed before, while it is true that the core components and key processes are based on the same information, studying the two separate profiles offers diagnostic information on how a director's behavior can be improved to achieve a more effective institution and, in turn, improve student performance. From the results obtained (Table 7), it can be seen that the level of performance of directors in the main components is distinguished, except for the component “connections to external community” that obtains a level of performance of competent.

On the other hand, when analyzing the performance level of the key processes (Table 8), we observe how the level of competence obtained is distinguished in all of them.

Conclusions

The evolution of virtual learning environments underlines the importance of the role of leaders in improving both the development of institutions and the achievement of proposed goals and objectives. Indeed, the success of these institutions is closely linked to meeting the needs of students and the demands of a society in permanent change. In this regard, it is essential that leaders understand these needs, so that they can provide opportunities to develop effective programs focused on both the student and the mission and academic culture of the institution.

In this research, through the application of the adaptation and translation of the VAL-ED for the university context, we have been able to advance a better understanding of the effectiveness of e-leadership behaviors focused on the learning of the managers of the Faculty of Education of the International University of La Rioja. Likewise, we have also been able to identify those areas of e-leadership behavior that represent areas of relative strength, and which represent areas of possible improvement on which further work is needed.

The main objective was to analyze the pedagogical e-leadership in online higher education in Spain through the application of the VAL-ED at the International University of La Rioja, specifically in the Faculty of Education. To do this, first we have sought, as we set out in the first objective, to identify the effectiveness of behavior in the director's leadership, taking the VAL-ED competency standards as a reference. Likewise, it has been possible to verify which areas of leadership behavior represent the greatest strengths, limitations and main needs for improvement.

Indeed, the data obtained allow us to affirm how both directors and supervisors, as well as faculty, have been able to evaluate the e-leadership behaviors exercised in a virtual teaching environment by managers in comparison with the VAL-ED competency standards, having obtained very positive results that show us how their leadership is oriented to students. Given the fact that in the second specific objective we set out to determine to what extent the three groups of respondents (faculty, supervisors and directors) coincide in the effective performance of the pedagogical leadership of the directors, it can be highlighted that no great differences have been found in the responses of effectiveness obtained from directors, supervisors and teachers; since the results of the resulting scores among these three groups of respondents were reasonably similar, this allows us to affirm that the data obtained are valid and in line with the reality of the e-leadership of directors.

Among the limitations that we find in this study, the first one that we should highlight is the sample size of the group of teachers, as although a response rate of 75% or more is considered high and therefore desirable by increasing the likelihood that the resulting assessment data will be representative of the interacting respondents with the director, the ideal would have been to have a 100% response rate, as it has happened with the rest of the groups (directors and supervisors).

On the other hand, it should also be noted that, when determining the level of effectiveness and performance of directors, we have worked with average scores. These, like any test score, are observed scores and are likely to have some associated error, although it should be added that the possible error in the VAL-ED is very low.

Likewise, we know that one of the characteristics that the VAL-ED has when applied in contexts of compulsory education (primary and secondary) is that it evaluates to what extent the leadership behavior of the director is effective compared to a national sample of principals. In this sense, as this study is the first in which the questionnaire is applied in a context of higher education, added to the fact that we do not have a sufficiently significant sample of results at the national level, we have not been able to make this comparison.

Lastly, and as we find in Palomino et al. (2022a), given the small number of works in this line, and in view of the results obtained, it is worth highlighting the need to go deeper into this area of research, that will lead to the efficiency and effectiveness of organizational development, as well as in the improvement of all dimensions of educational institutions of higher education. Therefore, the research work must continue observing the necessary scientific standards that ensure the quality of knowledge, in order to understand to what extent the influence of learning management in Higher Education is aimed in order to increase and improve the performance of its students, as well as the improvement of the quality of the institutions.

Among the main future lines of research that could be inspired by these results, on the one hand, is the investigation of the leadership models that are being carried out in educational institutions. On the other hand, it would also be necessary to analyze the role of the leader both in the promotion of student performance and in the increase of quality in higher education institutions.