ECET Continuous Improvement Plan

The Electrical Engineering Technology program of the ECET Department has an ongoing assessment and continuous improvement plan in place since 1995. The plan has gone through an evolutionary path and was refined during this time frame. The department has embraced the general philosophy of Outcome Based Education. In its current form it is designed to encompass all aspects of Outcome Based Assessment conforming to TAC/ABET’s model.

EET Program Assessment Methodology

The following brief expose to Outcome Based Education is presented here as a frame of reference and thankfully acknowledges the sources from the cited reference literature.

Outcome Based Education (OBE)

Outcome Based Education is the paradigm shift resulting from the failure of Traditional Education (TE). TE narrowly focused on the content and produced students with varying degrees of achievement levels (stratification of achievers). Thus this model did not produce learners, which could perform effectively in the work place. OBE has changed the focus of learning institutions from the content to the learner. According to William Spady, a major proponent of OBE, three goals drive this approach to creating academic curricula. 1) All students can learn and succeed, but may be not on the same day or in the same way. 2) Each success by a student breeds more success. 3) Academic institutions control the conditions of success.

Curriculum Design for OBE

OBE is a methodology of curriculum design and teaching that focuses on what students can actually do after they are taught. OBE focuses on these key questions as to:

1. What should the students learn?
2. What is the motivation for the students to learn it?
3. How can the academic institution and its resources help students learn it?
4. How will it be determined what the students have learned (assessment)?

Thus, the OBE’s instructional planning process is a reverse of that associated with traditional educational planning. The desired outcome is determined first and the curriculum, instructional materials and assessments are designed around to support and facilitate the intended outcome (Spady 1988; 1993). All curriculum and teaching decisions are made based on how best to facilitate the desired final outcome.

Essentials for OBE’s successful

Towers (1996) noted the following four points to the OBE system that are essential for its success:

1. Identification of learning material.
2. The student’s achievement is based on demonstrable measurables.
3. Multiple instructional and assessment strategies need to be utilized to meet the needs of each and every student.
4. Adequate time and needed assistance is to be provided so that each student can reach the maximum potential.

Why OBE?

The benefits of OBE are as follows:

OBE is able to measure—what the students are capable to perform – this goes much deeper at a cognitive level than traditional education system which determines whether or not students know some facts (content) or some predetermined mechanical process of doing a task. OBE on the other hand not only provides the students with facts (content) it takes the approach of problem solving, which calls for all the necessary permutations and combinations which the situation demands. Hence the student is intellectually engaged in solving problems which are quite varied and situational. OBE is not only interested in content but more so in context which lends the relevancy to the content itself. Classroom practices have to allow the students to demonstrate what they have learned in many different ways. Ideally, students should have an understanding of the content, context and must have internalized the subject matter at a cognitive level and have gained mastery over it. (Spady, 1995).

OBE goes beyond ‘structured tasks’ (e.g. memorization) by demanding that a student demonstrate his/her skills through more challenging tasks like designing and project building. They should engage in analysis and synthesis and apply it to problem solving. Many times they should utilize other partner’s resources in the equation of problem solving. The students learn the value of cooperation in place of mindless competition that is detrimental to team spirit and resources development, which are critical to complex problem situation and solution. Thus, OBE involves students in a complete course of learning—from developing their skills in designing to completing a whole process (Spady, 1994, 1995). OBE also identifies higher levels of thinking (e.g. creativity, ability to analyze and synthesize information, ability to plan and organize tasks). Such skills are emphasized especially when students are assigned to organize and work as a community or in teams to propose solutions to problems and market their solutions.

The Four Basic Principles of OBE

1. Clarity of focus about outcomes
*
• Always have the significant, culminating exit outcomes as the focus.
• Let the students know what they are aiming for.
  *There could be two types of outcomes: major ones such as the exit outcome of the course and minor ones that are developed by the instructor for achieving the instructional goals.
2. Designing backwards
• Design curriculum backward by using the major outcomes as the focus and linking all planning, teaching and assessment decisions directly to these outcomes.
3. Consistent, high expectations of success
• Set the expectation that OBE is for ALL learners.
• Expect students to succeed by providing them encouragement to engage deeply with the issues they are learning and to achieve the high challenging standard set (Spady, 1994b).
4. Expanded opportunity (Inclusive)
• Develop curriculum to give scope to every learner to learn in his/her own pace.
• Cater for individual needs and differences, for example, expansion of available time and resources so that all students succeed in reaching the exit outcomes.

Using Outcomes to Guide Instructional Planning

Instructional planning under OBE system takes four major steps:

1. Deciding on the outcomes

Outcomes and standards should be described in terms of three dimensions:

• CONTENT—simple to complex
• CONTEXT—simple to complex
• COMPETENCE—low to high

They should be arrived at through the input from the different constituencies, which the institution deals with.



2. Demonstrating outcomes Demonstration of the outcome is set forth by arriving at ‘benchmarks’ for each level of the program. Each of these benchmark is a skill-set that must be demonstrated by the student. Unlike the outcomes, the list of benchmarks is different in every level of the program of interest. Benchmarks should address and define specifically the goals of the curriculum and determine ways to assess whether students have reached these goals at that level of study.



3. Deciding on contents and teaching strategies Students and instructor should jointly develop the content and teaching strategies. Students should actively participate and decide in the process of learning. Classroom should provide experiences that students are going to encounter in the workplace. Empowering the student and engaging them in learning process is the goal and path to life-long learning.



4. Assessments in OBE The entire curriculum in OBE is driven by assessments that focus on well-defined learning outcomes and not primarily by factors such as what is taught, how long the student takes to achieve the outcomes or which path the student takes to achieve their target. The learning outcomes are set out on a gradation of increasing complexity that students are expected to master these outcomes sequentially. (Willis & Kissane1995) suggested two techniques for assessing students’ learning outcomes:
• ‘Standard-referenced assessment’ (similar to criterion-referenced assessment but with a clearer description of expected performance), and
• Student portfolios documenting their progress.


Given that assessments in OBE focus on the students’ learning outcomes (i.e. how much and how well the students have learnt), this could imply that students with different abilities will follow different paths to reach their goals and may finish at different times. This brings forth some questions on when and how often to carry out the assessments in a semester or how many attempts should a particular student be allowed to show her/his abilities.

In addition, as OBE requires ongoing feedback between the student and the lecturer, continuous assessments could help the lecturers determine the following:
• How to achieve the learning outcomes?
• What is the progress of particular students in the class?
• When to assess the students on how much they have learned?

Conclusion

According to Chandrama Acharya, OBE promises high level of learning for ALL students as it facilitates the achievement of the outcomes, characterized by its appropriateness to each learner’s development level and active and experienced-based learning. Moreover, knowing that this system is going to be used would also give students the freedom to study the content of the course in a way that helps them learn it. OBE must involve administrators, educators, parents, teachers and students (all constituencies) for successful implementation.

References

Spady, W. (1988). ‘Organizing for Results: The Basis of Authentic Restructuring and Reform’. Educational Leadership. Vol. 46, No. 2 pp. 4–8.
Spady, W. (1993). Outcome-based Education. Belconnen, ACT: Australian Curriculum Studies Association.
Spady, W. (1994a). ‘Choosing Outcomes of Significance’. Educational Leadership. Vol. 51, No. 6, pp. 18–22.
Spady, W. (1994b). Outcome-based Education: Critical Issues and Answers. Arlington, VA: American Association of School Administrators.
Spady, W. and Marshall, K. (1994). ‘Light, not Heat, on OBE’. The American School Board Journal. Vol. 181, pp. 29–33.
Spady, W. (1995). ‘We Need More ‘Educentric’ Standards’. Educational Leadership. Vol. 53, No. 1, pp. 82–84.
Towers, J.M. (1996). ‘An Elementary School Principal’s Experience with Implementing an Outcome-based Curriculum’. Catalyst for Change. Vol. 25, pp. 19–23.
Outcome Based Education (OBE): A New Paradigm for Learning, Ms Chandrama Acharya, Former Research Assistant, Center for the Development of Technological Leadership.

ABET’s Outcome Based Assessment Model

Outcome Based Assessment is the culminating part of Outcome Based Education (OBE). ABET’s model is as follows:

1. Each program has major defining constituencies.



2. The constituencies define the Program Education Objectives (long term and broad outcomes).



3. ABET assessment is based on Outcome Based Education’s methodology, and defines a through k components for the Program’s Outcomes as the General Criteria.



4. IEEE as the lead society for the Electrical Engineering Technology Program defines the program specific outcomes as the Program Criteria.



5. Each program designs a curriculum that incorporates the General Criteria and Program Criteria as the Program’s Outcome, defined in terms of the Program Learning Objectives.



6. Each program accomplishes Outcome Based Assessment as an integral part of the Outcome Based Education. This process utilizes multiple tools to assess and evaluate, which in turn translates into Continuous Improvement of Outcome Based Education.

Significant Constituencies of the Program

The following constituencies of the Electrical Engineering Technology Program dynamically define and shape the program’s content, context and currency:

1. Faculty
2. Student
3. Industrial Advisory Committee
4. Alumni
5. Employers
6. ABET – as the professional accreditation body
7. IEEE – as the lead professional society

The above constituencies play a direct role as depicted in Figure 1, in defining the EET Program’s Educational Objectives. The constituencies play a defining role in the EET Program’s Outcome Based Education and its Outcomes. The EET Program Outcomes are distributed throughout the EET curriculum.

The EET Program’s PEO and PO assessment process is elaborated in the following Figure.

EET Program Assessment and Evaluation

The EET Program has put forth the following process to establish and review the Program’s Education Objectives (long term objectives) and Program’s Outcomes (at the time of graduation) by utilizing the following tools.

Assessment Tools for Program Education Objectives (PEO)

1. Alumni Survey
2. Employers’ Survey
3. Input from Industrial Advisory Committee
4. Program Educational Objectives Students’ Survey
5. Faculty Annual Self-Assessment (faculty retreat)
6. Facilities and Resources Assessment (faculty retreat)

Assessment Tools for Program Outcomes (PO)

1. Course Embedded Assessment
2. Student Course Satisfaction Exit Survey
3. Cumulative GPA Index for Each Course
4. Electrical Engineering Technology -- Basics Exam
5. Senior Design Projects -- Index of Excellence
6. 1. Program TAC/ABET Accreditation
   2. Academic Review - Purdue University Calumet

PEO Assessment Tools – Protocols

1. PEO Assessment Tool: Alumni Survey Protocol

Measurable: Program’s adequacy toward stated PEOs and adequacy of PEOs.
Constituency: Alumni
Methodology of Data Gathering: Survey Form
Data collection frequency: Yearly
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and Inferential
Frequency of Analysis and Interpretation: Every three years
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible for this assessment tool: Professor Ashfaq Ahmed
Assessment Goal: 75% of the graduates surveyed would answer all survey items at a scale of 2.5 or better out of 4.0

2. PEO Assessment Tool: Employers’ Survey Protocol

Measurable: Program’s adequacy toward stated PEOs and adequacy of PEOs.
Constituency: Employers
Methodology of Data Gathering: Survey forms, including online ECET link and mailing
Data collection frequency: Once a year
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and Inferential
Frequency of Analysis and Interpretation: Once a year
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible for this assessment tool: Professor Essaid Bouktache
Assessment Goal: 60% of the received surveys will convey an average of 2.5/4

3. PEO Assessment Tool: Input from Industrial Advisory Committee Protocol

Measurable: Program’s adequacy toward stated PEOs and adequacy of PEOs
Constituency: Industrial Advisory committee
Methodology of Data Gathering: Input from minutes of the meeting
Data collection frequency: Every year
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and Inferential
Frequency of Analysis and Interpretation: Every year
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible: Professor Masoud Fathizadeh
Assessment Goal: Recommendations after faculty analysis are realized within the realm of available resources

Link to Electrical and Computer Engineering Technology Industrial Advisory Committee

Link to Electrical and Computer Engineering Technology Industrial Advisory Committee Meeting Minutes

4. PEO Assessment Tool: Program Educational Objectives Students’ Survey Protocol

Measurable: Program’s adequacy toward stated PEOs and adequacy of PEOs.
Constituency: Student Body
Methodology of Data Gathering: A representative sample (36 students) of student body is randomly chosen from 2nd year (4th semester), 3rd year and 4th year standing of EET Program
Data collection frequency: Spring of each year
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and Inferential
Frequency of Analysis and Interpretation: Every three years
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible: Professor Jai P. Agrawal
Assessment Goal: 50% of the surveyed items (1-10), would each have a mean of 2.5 or better or in other words each of the items surveyed would have a median of 2.5 or better

5. PEO Assessment Tool: Faculty Annual Self-Assessment Protocol

Measurable: Program’s adequacy toward stated PEOs and adequacy of PEOs
Constituency: Department’s Faculty
Methodology of Data Gathering: Annual Review
Data collection frequency: Once a year
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and Inferential
Frequency of Analysis and Interpretation: Once a year
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible: Professor Essaid Bouktache
Assessment Goal: 75% of the time the faculty self established goals are achieved at the level of “Meets All Expectation”

6. PEO Assessment Tool: Facilities and Resources Assessment Protocol

Measurable: Program’s adequacy toward stated PEOs and adequacy of PEOs.
Constituency: Department’s Faculty
Methodology of Data Gathering: Agenda line item during Annual Faculty Retreat
Data collection frequency: Every year
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and Inferential
Frequency of Analysis and Interpretation: Every year
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible: Professor Omer Farook
Assessment Goal: Faculty input is translated into Annual Capital Budget and Expenditure

PO Assessment Tools -- Protocols

1. PO Assessment Tool: Course Embedded Assessment Protocol

Measurable: Realization of course learning objectives and a – k ABET/Departmental Outcomes
Constituency: Students
Methodology of Data Gathering: Student work
Data collection frequency: Each semester for each course offered
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and/or inferential
Frequency of Analysis and Interpretation: Every semester
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Concerned faculty and the Quality Improvement committee in their respective discipline
Faculty Responsible: Professor Omer Farook
Assessment Goal: Each assessed item would have a score of 70% or better. Faculty would take corrective action to any item having a score of less than 70%

2. PO Assessment Tool: Student Course Satisfaction Exit Survey Protocol

Measurable: Realization of course learning objectives and a – k ABET/Departmental Outcomes
Constituency: Students and Faculty
Methodology of Data Gathering: Survey form completed at the conclusion of each class
Data collection frequency: Each Semester for each course offered
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and/or Inferential
Frequency of Analysis and Interpretation: Every Semester
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Concerned Faculty
Faculty Responsible: Professor Ashfaq Ahmed
Assessment Goal: Each semester 90% of the courses should have a rating at or above 2.5 out of 4

3. PO Assessment Tool: Course Cumulative GPA Index Protocol

Measurable: Realization of course learning objectives and a – k ABET/Departmental Outcomes
Constituency: Students and Faculty
Methodology of Data gathering: Data collected from each course at the end of the semester
Data collection frequency: Each Semester for each course offered
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and/or Inferential
Frequency of Analysis and Interpretation: Every Semester
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Concerned Faculty
Faculty Responsible: Professor Jai P. Agrawal
Assessment Goal: Each semester 90% of the courses should have a cumulative GPA Index at or above 2.5 out of 4

4. PO Assessment Tool: Electrical Engineering Technology Basics Exam Protocol

Measurable: The soundness of theoretical knowledge base will be measured by a comprehensive examination to be taken by all the graduating students of ECET at the associate level. This examination will comprise of: “the concept mapping“of all the core courses of ECET curriculum (Circuit theory, Digital Electronics, Analog Electronics and Computer Programming)
Constituency: Students
Methodology of Data Gathering: A computational examination
Data collection frequency: Each Semester
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and/or Inferential
Frequency of Analysis and Interpretation: Every two years
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible: Professor Chandra Sekhar
Assessment Goal: Each semester 55% of the ECET students who will take the comprehensive examination will score more than 60%

5. PO Assessment Tool: Senior Design Projects Index of Excellence Protocol

Measurable: The soundness of designing capability along with technical report writing and oral presentation skills of students will be measured by a panel made up of faculty and engineering staff from the industry. All ECET 491, Senior Design projects will be evaluated on the above mentioned criteria. The input of the panel will be cumulated into a 100 point index
Constituency: Students, Faculty and Industrial Advisory Body
Methodology of Data Gathering: All the members of the panel will fill a measurement rubric for all successful projects
Data collection frequency: Each Semester ECET 491 is offered
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and/or Inferential
Frequency of Analysis and Interpretation: Every two years
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible: Professor Omer Farook
Assessment Goal: Each semester the cumulative index of excellence for the Senior Design in the Electrical Engineering Technology will be better than 75 %

6.a PO Assessment Tool: Program TAC/ABET Accreditation

Measurable: Program Accreditation from TAC/ABET – indicating that the program satisfies the published criteria of TAC/ABET
Constituency: TAC/ABET
Methodology of Data Gathering: Self-study report and final TAC/ABET outcome
Data collection frequency: Once every six years
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and/or Inferential
Frequency of Analysis and Interpretation: Every six years
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible: Professor Essaid Bouktache
Assessment Goal: Get the maximum years of accreditation (six years)

6.b PO Assessment Tool: Purdue University Calumet Wide Academic Review Protocol

Measurable: Program viability toward the department’s stated goals
Constituency: University-wide faculty committee outside of the department
Methodology of Data gathering: Comprehensive elf-study report on program evaluation
Data collection frequency: Once every five years
Data collection responsibility: Department Assessment Committee
Statistical Analysis: Descriptive and/or Inferential
Frequency of Analysis and Interpretation: Every five years
Responsibility for Analysis, Interpretation and Implementation resulting in Continuous Improvement: Department Faculty
Faculty Responsible: Professor Essaid Bouktache
Assessment Goal: Address and resolve all the stated recommendations