The benefit to the engineering community would be a reduction of process overlaps, significant saving in time and effort while ensuring both processes occur in the same time period, writes Maria Kyne.
Formative development
The definition of the fundamental purpose of engineering education is given in the International Engineering Alliance Graduate Attributes and Professional Competencies document as ‘to build a knowledge base and attributes to enable the graduate to continue learning and to proceed to formative development that will develop the competencies required for independent practice’ (International Engineering Alliance (IEA), 2013).
Professional bodies measure the quality of engineering education in two ways. Outcomes evidence based criteria are used to evaluate engineering education programmes and competency based standards are used to assess if engineers can gain professional recognition.
The systematic development of robust quality assurance procedures in higher education was heralded in the 1992 Green Paper on Education (Department of Education and Science, 1992) and expanded in the 1995 White Paper on Education (Department of Education and Science, 1995).
Quality Assurance in Higher Education is the totality of systems, resources and information devoted to maintaining and improving the quality and standards of teaching, scholarship and research and of students' learning experience (The Quality Assurance Agency in Higher Education, 1998).
Quality assurance guidelines
Irish institutes of technology hold delegated authority to make their own awards and are obliged to have regard to quality assurance guidelines issued by Quality and Qualifications Ireland (QQI) (Quality and Qualifications Ireland, 2016).
All registered education providers are required to conduct cyclical programmatic reviews of their programmes. In addition, standards and guidelines for quality assurance in the European Higher Education Area (ESG) requires that higher education institutions should monitor and periodically review their programmes to ensure that they achieve the objectives set for them and respond to the needs of students and society (European Association for Quality Assurance in Higher Education (ENQA), 2015).
All programmes of study in institutes of technology in Ireland are subjected to internal programmatic review in five yearly cycles to ensure that the programmes meet the quality assurance standards and are fit for purpose (Quality and Qualifications Ireland, 2016).
In addition, engineering and construction programmes undergo voluntary external accreditation by their respective professional bodies (Quality and Qualifications Ireland, 2019).
Both processes differ in their focus and intent and the preparation required by the programme teams and managers. The two processes emphasise different aspects of engineering education (Quality and Qualifications Ireland, 2017).
Utilised worldwide
From the research literature, it has emerged that these assessment types are utilised worldwide, in varying ways and in regular cycles, for the quality assurance of engineering programmes. Both the programmatic review and accreditation processes have evolved and diverged over time.
The programmatic review process is normally conducted on a faculty or department wide basis and involves a root and branch examination of programmes of study and how they have been delivered in the previous five years and how they plan to be delivered in the subsequent five years.
Industry and stakeholder consultation is a critical part of the process. Programmes are changed to include new technologies and new delivery methods while ensuring that graduates have the requisite skills and competencies to prepare them for the world of work.
Accreditation of engineering programmes by professional bodies such as Engineers Ireland (EI), the Society of Chartered Surveyors Ireland (SCSI) and others, are a vital part of ensuring that programmes are fit for purpose and that graduates have the requisite skills to be able to participate fully in their chosen profession (Engineers Ireland, 2014) (The Royal Institution of Chartered Surveyors (RICS), 2019).
Engineers Ireland has formally accredited all university and institutes of technology engineering programmes in Ireland since 1982.
Satisfy appropriate criteria
Engineering education programmes which satisfy the appropriate criteria laid down in the accreditation criteria for professional titles documents are deemed to meet the education standard required of individuals seeking one of the registered titles of Chartered Engineer, associate engineer and engineering technician (Engineers Ireland, 2014).
The accreditation process, as laid down in the document is consistent with international best practice and this is verified by their inclusion in international mutual recognition agreements, such as the Washington Accord. Engineers Ireland has also issued a supporting guidance document titled 'Procedure for Accreditation of Engineering Education Programmes' (Engineers Ireland, 2015).
The purpose of accreditation is to evaluate engineering education programmes against standards agreed upon and accepted by the international academic community and relevant industry stakeholders (Aqlan, et al., 2010).
The accreditation process is voluntary and usually embraces a combination of self-evaluation, external peer review based on a site visit, recommendation by the visiting panel and the final decision is made by the responsible accreditation/education board.
The focus of the accreditation process has changed significantly in the last 10 years towards the measurement of student achievement of learning outcomes.
Driving force
According to the research literature, this new accreditation process focus has gained worldwide acceptance and is a driving force for ensuring the quality of engineering education programmes.
The challenges to be overcome by this accreditation policy implementation include the ability to assess programme outcomes, workload and inconsistencies between evaluators (Patil & Codner, 2007).
Faculty staff have come to view the programmatic review process as principally a review of the faculty/department and the accreditation process as a more rigorous review of the programme content.
In engineering education quality assurance there are two main powerbrokers, the state and the professional bodies, acting as gatekeepers and controllers for the roll out of policy admission to the engineering profession.
The processes have a gatekeeper function where admission to a professional elite is controlled by adherence to the relevant policies and procedures.
In some countries, accreditation is conducted by a government organisation. In others, the quality assurance process is independent of government and is performed by private companies or professional bodies (Aqlan, et al., 2010).
In the US, ABET evaluates engineering education programmes and uses the ECriteria 2000 as the basis of their participation in international multinational agreements and mutual recognition agreements (Washington Accord).
Bologna Declaration
In Europe, there are many policy developments including the Bologna Declaration. Guidelines for quality assurance have been developed by the European Association for Quality Assurance in Higher Education (ENQA, 2015).
The establishment of the European Federation of National Engineering Associations (FEANI), the European Network for Accreditation of Engineering Education (ENAEE) and the development of EUR-ACE® has created a common approach to accreditation and assists in simplifying different systems (FEANI, 2019) (ENAEE, 2019).
In Asia, Australia and New Zealand have led the development of accreditation processes and were founder members of the Washington Accord. Some other countries are also members of the accord (Japan, Malaysia, Taiwan, China etc.) (Patil & Codner, 2007).
The programmatic review process is a European and national driven process whereas the engineering programme accreditation process has been developed by a national policy community (Engineers Ireland) but influenced by global policy communities (International Engineering Alliance, etc.).
The peer review aspect of the accreditation process brings a collaborative dimension to the process as well as participation on the decision making structures (accreditation/education boards).
The benefits of successful achievement of programmatic review and accreditation for the educational provider and graduates include public accountability, guarantee of quality, academic reputation, global professional recognition and registration, international mobility, academic improvement and educational competitiveness.
Gatekeepers to engineering profession
Significant benefits also accrue to the professional bodies who remain the gatekeepers to the engineering profession.
Professional body accreditation policies cannot be enabled without engagement with engineering education programmes and they in turn need the seal of accreditation so that their graduates can be elected into a professional engineering association.
The pursuit of accreditation has become mandatory for Higher Education Institutes as the consequences of not being accredited are dire for graduates who would not be able to practice as professional engineers (Said, et al., 2013).
I am currently studying for a PhD and my research question explores the possibility of the alignment or combination of the programmatic review and accreditation quality assurance processes for engineering education programmes in Ireland.
This alignment/combination could then allow for the establishment of a single collaborative quality assurance process for engineering education or facilitate sequential occurrence of the processes within the same timeframe. My research is supervised by Professor Merrilyn Goos and Dr Peter Tiernan, University of Limerick.
Gain insights from experts
As the research is designed as a qualitative study to gain insights from experts, the design philosophy supporting this research includes a pragmatic paradigm with a subjective ontology allowing multiple realities, an interpretative epistemology and axiology for value laden interpretation of qualitative research, using an adopted Delphi technique for data collection and the constructivist grounded theory to support the analysis of the data.
The characteristics of these methodological approaches were examined to ensure that they were all compatible for this research methodology.
Significant consultation has taken place with the gatekeepers of these processes. The Technological Higher Education Association (THEA) was established in the early 2000s to represent the institute of technology sector. Under THEA, the Council of Heads of School of Engineering (COHSE) was established.
Incorporation of the programmatic review process and accreditation process into a single quality assurance process has long been an ambition of the COHSE.
The author prepared a discussion document and comparison analysis of the two processes in consultation with COHSE. The position paper concluded that there is considerable overlap between the programmatic review and accreditation processes and some realignment/amalgamation of the processes would achieve the same outcomes.
Three COHSE representatives met with the THEA Council of Registrars and with the registrar of Engineers Ireland who agreed in principle with the approach and recommended further consultation with QQI.
The author met with QQI and the registrar of Engineers Ireland to consider if it is possible/practical to align the objectives of the programmatic review and Engineers Ireland accreditation processes.
The researcher prepared 24 triangulation documents comparing the QQI engineering award standards, the QQI professional award type descriptors and the Engineers Ireland accreditation criteria.
This allowed for comparison across the three engineering professional titles, their equivalent National Framework of Qualifications levels for the three strands of knowledge, skill and competence and the five sub-strands of mathematics and sciences, design and development, information technology, business context and engineering practice.
Even though there are differences in wording between the standards, there is more than 90 per cent alignment between all three sets of objectives in terms of their intent.
Action research intervenes in work practices to achieve change and improvement. The Delphi technique utilises action research to achieve consensus by using a series of rounds.
Data collection and analysis proceeds in an iterative process until consensus/theoretical saturation is reached where information is fed back to the research participants in a controlled manner. The constructed knowledge reflects both the researcher’s and participant’s views of the research area under investigation.
The main stages of the in-depth research are as follows:
1.) Delphi technique round 1 – Semi-structured interviews
2.) Delphi technique round 2 – Structured questionnaire using the findings in round 1
3.) Delphi technique round 3 – Semi-structured interviews using the findings in round 2.
Findings to date
Twenty six semi-structured interviews for the Delphi technique Round 1 were held with a predetermined multi-level expert group who had considerable knowledge and experience of the two quality assurance processes.
The comparative analysis was the basis on which the first round of questions were created. A focus group meeting was held with engineering staff from Limerick Institute of Technology to refine the questions for the Round 1 interviews.
The Round 1 findings have identified that the research participants are very supportive of the possibility of aligning/combining the two quality assurance processes.
Table 1: Round 1 table of overarching themes.
Seventeen themes and categories that are likely to hinder the possibility of bringing the processes into closer alignment were identified and categorised into those relating to the existing processes and those relating to new revised process(es) as shown in the table below.
The structured questionnaire for the Delphi technique Round 2 was created directly from the seventeen overarching themes emerging from the Round 1 interviews. Each question had a number of sub-questions.
The questionnaire, consisting of 83 sub-questions, was sent to all 26 participants from Round 1 and 24 participants completed the questionnaire. The tables below gives a sample for one of the theme areas.
For each sub-question a deeper analysis of participant answers was undertaken by group type and engineering discipline to compare the responses by the various categories of participants: registrars, professional body registrars, heads of faculty/school from both mechanical/electrical and civil engineering disciplines and heads of department and staff from the engineering discipline areas.
Table 2: Round 2 table of responses to the revised process – method of aligning/combining theme.
Table 3 illustrates that the method of alignment/combination is still unclear. Round 2 has identified other aspects of the processes where clear protocols need to be established between the gatekeepers and the higher education institutions at a high level.
Round 2 has agreed the findings from round 1 as the participants agreed or strongly agreed with 75 per cent of the sub-questions, disagreed or strongly disagreed with 11 per cent of the sub-questions and did not agree or disagree with 14 per cent of the sub-questions.
The Round 3 semi-structured interview questions will be generated directly from the outputs of the questionnaire from Round 2 and will assist in finalising the outcomes of the research. Some of the unresolved themes include:
• Mandatory or voluntary accreditation process
• Method of alignment/combination
• Synchronising of the review cycles
• Independence of the process outcomes (validation and accreditation)
• Sharing of responsibility
• Report generation and sign-off.
Table 3: Round 2 table of responses to the revised process – method of aligning/combining theme – group type and engineering discipline.
Conclusion
In institutes of technology there are many methods used to measure the quality assurance of engineering education programmes but the two major cumbersome processes are programmatic review and accreditation. Both processes differ in focus and intent but have considerable overlaps.
This research explores the possibility of the alignment or combination of the programmatic review and accreditation quality assurance processes for engineering education programmes in Ireland.
The research is designed to gain the insights from experts on how improvements to the management or scheduling of the processes could be achieved to enable the alignment/combination of the two processes.
The main themes and categories have been identified and are being considered in an iterative cycle to achieve consensus.
The benefit to the engineering community would be a reduction of process overlaps, significant saving in time and effort while ensuring both processes occur in the same time period.
Acknowledgement
I would like to acknowledge the contribution of all the research participants, members of the council of heads of School of Engineering, Council of Registrars, registrar of Engineers Ireland and QQI to the content of this paper.
References
1.) Aqlan, F, AL-Araidah, O. & Al-Hawari, T., 2010. Quality assurance and accreditation of engineering eduation in Jordan. The European Journal of Engineering Education Volume 35, Number 3, pp. 311-323.
2.) Department of Education and Science, 1992. Education for a Changing World - Green Paper on Education. Dublin: The Stationary Office.
3.) Department of Education and Science, 1995. Learning for Life. White Paper on Education. Dublin: The Stationary Office.
4.) ENAEE, 2019. The European Network for Accreditation of Engineering Education. [Online]
Available at: https://www.enaee.eu
5.) Engineers Ireland, 2014. Accreditation Criteria for Professional Titles. [Online]
Available at: http://www.engineersireland.ie
6.) Engineers Ireland, 2015. Procedure for the Accreditation of Engineering Education Programmes. [Online] Available at: http://www.engineersireland.ie
7.) European Association for Quality Assurance in Higher Education (ENQA), 2015. Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG), Brussels, Belgium.
8.) FEANI, 2019. European Federation of National Engineering Associations. [Online]
Available at: https://www.feani.org
9.) International Engineering Alliance (IEA), 2013. Graduate Attributes and Professional Competencies. [Online] Available at: http://www.ieagreements.org
10.) Patil, A & Codner, G, 2007. Accreditation of Engineering Education: Review, observations and proposal for a global accreditation. European Journal of Engineering Education, 32(6), pp. 639-651.
11.) Quality and Qualifications Ireland, 2016. Core Statutory Quality Assurance Guidelines. [Online]
Available at: http://www.qqi.ie
12.) Quality and Qualifications Ireland, 2016. Sector Specific - Insitute of Technology QA Guidelines. [Online] Available at: http://www.qqi.ie
13.) Quality and Qualifications Ireland, 2017. Professional Body Accreditation in Higher Education Institutions in Ireland. [Online] Available at: http://www.qqi.ie
14.) Quality and Qualifications Ireland, 2019. Accreditation Approval of Higher Education Programmes by Professional Bodies: QQI Insights. [Online] Available at: http://www.qqi.ie
15.) Said, S. M. et al., 2013. Accreditation of Engineering Programmes: an evaluation of current practices in Malaysia. International Journal of Technology and Design Education, Volume 23, pp. 313-328.
16.) The Quality Assurance Agency in Higher Education, 1998. Quality Assurnace : a new approach, London: Higher Quality, the bulletin of the QAA No.4.
17.) The Royal Institution of Chartered Surveyors (RICS), 2019. RICS Global Accreditation - Policy and Process. [Online] Available at: http://www.rics.org
Author: Maria Kyne is dean of the Faculty of Applied Science, Engineering and Technology at Limerick Institute of Technology.