In 2009, IUPUI faculty members who teach undergraduates developed course matrices that included all courses in each undergraduate program. The matrices (seeIUPUI PUL Matrix) list each course by name and note which Principles of Undergraduate Learning (PULs) receive a major emphasis (rating of 3). Other PULs receiving a lesser degree of emphasis were rated by the faculty as having moderate (2) or minor emphasis (1). The next step is to assess student learning of the designated PULs in each course. A plan/schedule to assess at least one section of each course over the next five years is now available. If you are scheduled to teach one of the identified sections, you will be asked to assess individual student attainment of the desired level of expertise for the PULs emphasized in the class and to submit these data with your class grades at the end of the semester. The PUL data will be stripped of all information identifying individual students and faculty members and tabulated for all courses under evaluation. The resulting data will be analyzed and become part of the data set used for the self-study report submitted to the Higher Learning Commission of the North Central Association.
FAQs
Frequently Asked Questions Concerning
Assessment of Student Attainment of the Knowledge and Skills
- How are PULs 1a, 1b, and 1c defined? They aren't the same as PULs 1a, 1b, 1c, 1d, and 1e in the official PUL descriptions.The PULs have not been defined precisely at the campus level because the definitions are expected to differ by discipline. Faculty in each discipline define the PULs as they pertain to that discipline.
- If PUL 1 is designated for my course, should I incorporate 1a, 1b, and 1c in my course?Because PUL 1 covers multiple aspects of the development of communication skills—written, oral, and quantitative—it may not be possible to expose the students to each facet of PUL 1 in a meaningful way within a given course. Thus, faculty may select 1a, 1b, or 1c.
- How does the meaning of a "3" (effective) rating in an introductory course compare to a "3" in an advanced course? Are the scales the same at both levels or is there a different scale at each level?The scales are different. An upper-level course should require a more advanced skill level than an introductory one.
- How will I report my PUL evaluations?At the end of each term, PUL evaluation rosters will be created in addition to your final grade roster. Once created, there will be a link from Oncourse labeled SIS PUL Evaluation (under the link for your final grade roster). You may also get to the PUL evaluation directly from One. Go to: Services > Faculty Systems > Go to Faculty Center and select PUL Evaluation.
- Will everyone use SIS to report student attainment of the PULs?Not everyone is using SIS. One school already had another detailed and comprehensive assessment process in place. Users of the ePortfolio may substitute data on learning derived from the ePortfolio on a case-by-case basis.
- How do I change the information about the PULs that is currently listed in the PUL matrix for my course?Each school has designated a person, usually the associate dean for undergraduate education, as the official PUL representative. That designee communicates with the school's class scheduling officer (the individual within the school who works with the registrar's office to build the class schedule each term). As part of the normal schedule building process, the academic unit scheduling officer provides any PUL updates to the registrar's office, where the changes will be applied.
- Will students see the PUL ratings their instructors have assigned?For purposes of developing data sets for institutional review, student identities will not be connected with their ratings. Individual instructors (or program/department faculty as a group) may decide for themselves whether or not to share PUL data with students.
- Should I report on my methods for arriving at the student PUL ratings (very effective, effective, somewhat effective, not effective)? How much student documentation should I keep? Will I be asked to provide evidence at some future time?While such reporting is not required, written assessment criteria allow the instructor to be consistent over time. If specific rubrics are adopted at the course or department level for courses taught by multiple instructors, then changes in overall student attainment of the PULs will be easier to identify and to track. Currently, there are no set guidelines for what or how much documentation should be kept. This decision is at the discretion of the instructor (or program faculty as a group), and documentation may be as simple as collecting PDFs of exemplars of outstanding or substandard work. Currently, there are no plans to require that instructors provide evidence of their assessment methods.
- Should the grade in the course and the PUL rating correspond?The grade and the student's PUL attainment may or may not correspond to each other. For instance, a student may perform at an acceptable level overall in a course because she scored well on the mid-term and final, which were in a multiple-choice format. But, for example, if "core communication skills" was given a strong emphasis and the student's writing indicated an unacceptable level of performance as evidenced by numerous punctuation or grammatical errors in one or more assignments, then she may be assessed as not having attained the level of performance required to master PUL 1. On the other hand, if the student excels in all course outcomes, the grade and PUL rating could correspond, again based on the mechanics of the course.
Curricular Uses of the PUL Evaluation Process
- Do I need to put information in my syllabus about how I am assessing student PUL attainment?
While including this information is not mandatory, the syllabus ideally will communicate the instructor's expectations for the students who have enrolled. Well-informed students are less likely to be disgruntled at the end of a class if they are fully aware of what and how their work will be evaluated, whether it be in the form of a class grade or an assessment of their level of skill with a particular PUL. This information will also allow students to develop a sense of where they may have deficiencies and allow them time to remedy any lack of attainment.
- Will I need to make major changes in my course instruction and assessment to meet the PUL reporting requirements?
That is a decision that you and other faculty in your program should make. Ideally, major changes in the design of a course or in assessment procedures will not be needed to meet the requirements. Because the PULs are the foundation for undergraduate learning at IUPUI regardless of discipline, instructors have been expected to incorporate and to evaluate student learning of them in their courses for the past decade. The new requirements formalize this process and ask faculty to provide data on student achievement of the knowledge and skills embodied in the PULs.
- How do we leverage existing, discipline-specific assessment efforts to meet campus needs for PUL assessment without duplicating work?
The best approaches are likely to depend on the nature of the disciplinary outcomes that have been defined for courses and programs and on the assessment methods already being used. A first step for most programs will be to map discipline-specific outcomes to relevant PULs, where possible. (It is likely that some disciplinary outcomes will correspond well with one or more PULs, while other disciplinary outcomes may not be directly related to any PUL.) Then, assessment of the discipline-specific outcomes can also provide information on students' attainment of the corresponding PULs.
- How will assessment of PULS affect faculty evaluations?
The assessment data being gathered on the PULs and on students' achievement of those PULs will be tabulated and analyzed at the school and institutional levels and will not be part of faculty evaluation. Faculty themselves will analyze patterns of student strength and weakness across courses and curricula to determine where pedagogical or curricular changes may be needed to assure that students are developing mastery of PUL-related learning as they progress toward a degree.
Rubrics and Other Helpful Examples
- Should I create a rubric for the major, moderate, and minor PULs designated in my course?
Using a rubric for each PUL may make it easier to determine whether students did, in fact, attain the expected level of performance, but rubrics are not required.
Accreditation
- What role will individual faculty members play in the NCA accreditation process?
The role of individual faculty members in the North Central Association's Higher Learning Commission (HLC) reaccreditation process will vary. All faculty members are expected to be knowledgeable about the accreditation process. To expedite development of the institutional self-study, many faculty members will be asked to evaluate student learning related to the PULs taught in their courses. These evaluations will occur according to schedules developed by each school. During the site visit in 2012, individual faculty members may be invited for an interview with the HLC site visit team on any number of topics under consideration by the team. Keeping updated on the progress of campus preparations for the visit will enable faculty to answer the visitors' questions confidently.
Faculty members are encouraged to contact their school PRAC representatives (a list of current PRAC members can be found at IUPUI PRAC) or the Center for Teaching and Learning for help with combining existing assessment approaches with PUL assessment in an efficient way.
If you have further questions pertaining to the Principles of Undergraduate Learning, please contact Richard Jackson (rjackson@iupui.edu).