Issuu

NSSE 2016 Engagement Indicators

University of Houston-Downtown

About

NSSE 2016 Engagement Indicators

About

This Report

Your Engagement Indicators Report

Engagement Indicators (EIs) provide a useful summary of the detailed information contained in your students’ NSSE responses. By combining responses to related NSSE questions, each EI offers valuable information about a distinct aspect of student engagement. Ten indicators, based on three to eight survey questions each (a total of 47 survey questions), are organized into four broad themes as shown at right.

Report Sections

Overview (p. 3)

Theme Reports (pp. 4-13)

Comparisons with HighPerforming Institutions (p. 15)

Detailed Statistics (pp. 16-19)

Interpreting Comparisons

Theme

Academic Challenge

Learning with Peers

Experiences with Faculty

Campus Environment

Engagement Indicator

Higher-Order Learning

Reflective & Integrative Learning

Learning Strategies

Quantitative Reasoning

Collaborative Learning

Discussions with Diverse Others

Student-Faculty Interaction

Effective Teaching Practices

Quality of Interactions

Supportive Environment

Displays how average EI scores for your first-year and senior students compare with those of students at your comparison group institutions.

Detailed views of EI scores within the four themes for your students and those at comparison group institutions. Three views offer varied insights into your EI scores:

Mean Comparisons

Straightforward comparisons of average scores between your students and those at comparison group institutions, with tests of significance and effect sizes (see below).

Score Distributions

Box-and-whisker charts show the variation in scores within your institution and comparison groups.

Performance on Indicator Items

Responses to each item in a given EI are summarized for your institution and comparison groups.

Comparisons of your students’ average scores on each EI with those of students at institutions whose average scores were in the top 50% and top 10% of 2015 and 2016 participating institutions.

Detailed information about EI score means, distributions, and tests of statistical significance.

Mean comparisons report both statistical significance and effect size. Effect size indicates the practical importance of an observed difference. For EI comparisons, NSSE research has concluded that an effect size of about .1 may be considered small, .3 medium, and .5 large (Rocconi & Gonyea, 2015). Comparisons with an effect size of at least .3 in magnitude (before rounding) are highlighted in the Overview (p. 3).

EIs vary more among students within an institution than between institutions, like many experiences and outcomes in higher education. As a result, focusing attention on average scores alone amounts to examining the tip of the iceberg. It’s equally important to understand how student engagement varies within your institution. Score distributions indicate how EI scores vary among your students and those in your comparison groups. The Report Builder—Institution Version and your Major Field Report (both to be released in the fall) offer valuable perspectives on internal variation and help you investigate your students’ engagement in depth.

How Engagement Indicators are Computed

Each EI is scored on a 60-point scale. To produce an indicator score, the response set for each item is converted to a 60-point scale (e.g., Never = 0; Sometimes = 20; Often = 40; Very often = 60), and the rescaled items are averaged. Thus a score of zero means a student responded at the bottom of the scale for every item in the EI, while a score of 60 indicates responses at the top of the scale on every item.

For more information on EIs and their psychometric properties, refer to the NSSE website: nsse.indiana.edu

Institutional Research Annual Forum, Denver, CO.

Rocconi, L., & Gonyea, R. M. (2015). Contextualizing student engagement effect sizes: An empirical analysis. Paper presented at the Association for

Engagement Indicators: Overview

NSSE 2016 Engagement Indicators

University of Houston-Downtown

Engagement Indicators are summary measures based on sets of NSSE questions examining key dimensions of student engagement. The ten indicators are organized within four broad themes: Academic Challenge, Learning with Peers, Experiences with Faculty, and Campus Environment. The tables below compare average scores for your students with those in your comparison groups.

Use the following key:

▲ Your students’ average was significantly higher (p < .05) with an effect size at least .3 in magnitude.

△ Your students’ average was significantly higher (p < .05) with an effect size less than .3 in magnitude. No significant difference.

▽ Your students’ average was significantly lower (p < .05) with an effect size less than .3 in magnitude.

▼ Your students’ average was significantly lower (p < .05) with an effect size at least .3 in magnitude.

First-Year Students

Theme Engagement Indicator

Higher-Order Learning

Reflective & Integrative Learning

Learning Strategies

Quantitative Reasoning

Learning with Peers

Collaborative Learning

Discussions with Diverse Others

Student-Faculty Interaction

Effective Teaching Practices

Quality of Interactions

Supportive Environment

Seniors

Theme Engagement Indicator

Higher-Order Learning

Reflective & Integrative Learning

Learning Strategies

Quantitative Reasoning

Collaborative Learning

Discussions with Diverse Others

Student-Faculty Interaction

Effective Teaching Practices

Quality of Interactions

Supportive

NSSE 2016 Engagement Indicators

Academic Challenge University of Houston-Downtown

Academic Challenge: First-year students

Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote student learning by challenging and supporting them to engage in various forms of deep learning. Four Engagement Indicators are part of this theme: Higher-Order Learning, Reflective & Integrative Learning, Learning Strategies, and Quantitative Reasoning. Below and on the next page are three views of your results alongside those of your comparison groups.

Mean Comparisons

Engagement Indicator

Higher-Order Learning

Reflective & Integrative Learning

Learning Strategies

Quantitative Reasoning

first-year students

with

Notes: Results weighted by institution-reported sex and enrollment status (and institution size for comparison groups); Effect size: Mean difference divided by pooled standard deviation; Symbols on the Overview page are based on effect size and p before rounding; *p < .05, **p < .01, ***p < .001 (2-tailed).

Score Distributions

Notes: Each box-and-whiskers chart plots the 5th (bottom of lower bar), 25th (bottom of box), 50th (middle line), 75th (top of box), and 95th (top of upper bar) percentile scores. The dot represents the mean score. Refer to Detailed Statistics for your institution’s sample sizes.

NSSE 2016 Engagement Indicators

Academic Challenge

University of Houston-Downtown

Academic Challenge: First-year students (continued)

Performance on Indicator Items

The table below displays how your students responded to each EI item, and the difference, in percentage points, between your students and those of your comparison group. Blue bars indicate how much higher your institution's percentage is from that of the comparison group. Orange bars indicate how much lower your institution's percentage is from that of the comparison group.

Percentage point difference a between your FY students and UHD

Percentage responding "Very much" or "Quite a bit" about how much coursework emphasized…

Applying facts, theories, or methods to practical problems or new situations

Analyzing an idea, experience, or line of reasoning in depth by examining its parts

Evaluating a point of view, decision, or information source

Forming a new idea or understanding from various pieces of information

Percentage of students who responded that they "Very often" or "Often"…

Combined ideas from different courses when completing assignments

Connected your learning to societal problems or issues

2c. Included diverse perspectives (political, religious, racial/ethnic, gender, etc.)

Examined the strengths and weaknesses of your own views on a topic or issue

2e. Tried to better understand someone else's views by imagining how an issue looks from his or her perspective

Learned something that changed the way you understand an issue or concept

Connected ideas from your courses to your prior experiences and knowledge

Percentage of students who responded that they "Very often" or "Often"…

Reviewed your notes after class

Summarized what you learned in class or from course materials

Reasoning

Percentage of students who responded that they "Very often" or "Often"…

6a. Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.)

6b. Used numerical information to examine a real-world problem or issue (unemployment, climate change, public health, etc.)

Evaluated what others have concluded from numerical information

a. Percentage point difference = Institution percentage – Comparison group percentage. Because results are rounded to whole numbers, differences of less than 1 point may or may not display a bar. Small, but nonzero differences may be represented as +0 or -0.

Academic Challenge: Seniors

NSSE 2016 Engagement Indicators

Mean Comparisons

Engagement Indicator

Higher-Order Learning Reflective & Integrative Learning

Strategies

Quantitative Reasoning

Score Distributions

Higher-Order Learning

Learning Strategies

& Integrative

Quantitative Reasoning

NSSE 2016 Engagement Indicators

University of Houston-Downtown

Academic Challenge: Seniors (continued)

Performance on Indicator Items

Percentage point difference a between your seniors and Southwest

Percentage responding "Very much" or "Quite a bit" about how much coursework emphasized…

Applying facts, theories, or methods to practical problems or new situations

Analyzing an idea, experience, or line of reasoning in depth by examining its parts UHD

Evaluating a point of view, decision, or information source

Forming a new idea or understanding from various pieces of information

& Integrative Learning

Connected your learning to societal problems or issues

2c. Included diverse perspectives (political, religious, racial/ethnic, gender, etc.) in course discussions or assignments

Examined the strengths and weaknesses of your own views on a topic or issue

Tried to better understand someone else's views by imagining how an issue looks from his or her perspective

Learned something that changed the way you understand an issue or concept

Connected ideas from your courses to your prior experiences and knowledge

Percentage of

"Very often" or "Often"…

Identified key information from reading assignments

Reviewed your notes after class

Summarized what you learned in class or from course materials

Percentage of students who responded that they "Very often" or "Often"…

6a. Reached conclusions based on your own analysis of numerical information (numbers, graphs, statistics, etc.)

Used numerical information to examine a real-world problem or issue (unemployment, climate change, public health, etc.)

Evaluated what others have concluded from numerical information

NSSE 2016 Engagement Indicators

Learning with Peers

University of Houston-Downtown

Learning with Peers: First-year students

Collaborating with others in mastering difficult material and developing interpersonal and social competence prepare students to deal with complex, unscripted problems they will encounter during and after college. Two Engagement Indicators make up this theme: Collaborative Learning and Discussions with Diverse Others. Below are three views of your results alongside those of your comparison groups.

Mean Comparisons

Engagement Indicator

Learning

Discussions with Diverse Others

Score Distributions

Collaborative Learning

Discussions with Diverse Others

Performance on Indicator Items

Percentage point difference a between your FY students and

Collaborative Learning

1e. Asked another student to help you understand course material

1f. Explained course material to one or more students

1g. Prepared for exams by discussing or working through course material with other

1h. Worked with other students on course projects or assignments

Discussions with Diverse Others

8a.

Learning with Peers: Seniors

NSSE 2016 Engagement Indicators

Learning with Peers University of Houston-Downtown

Mean Comparisons

Engagement Indicator

Discussions with Diverse Others

Your seniors compared with UHD

Score Distributions

Collaborative Learning

Notes: Each box-and-whiskers chart plots the 5th (bottom of lower bar), 25th (bottom of

scores. The dot represents the mean score. Refer to Detailed Statistics for your institution’s sample sizes.

Discussions with Diverse Others

and

Performance on Indicator Items Collaborative Learning

Percentage of students who responded that they "Very often" or "Often"…

1e. Asked another student to help you understand course material

Explained course material to one or more students

1g. Prepared for exams by discussing or working through course material with other

1h. Worked with other students on course projects or assignments

Discussions with Diverse Others

Percentage of students who responded that they "Very often" or "Often" had discussions with…

8a.

your

and

a. Percentage point difference = Institution percentage – Comparison group percentage. Because results are rounded to whole numbers,

display a bar. Small, but nonzero differences may be represented as +0 or -0.

NSSE 2016 Engagement Indicators

Experiences with Faculty University of Houston-Downtown

Experiences with Faculty: First-year students

Students learn firsthand how experts think about and solve problems by interacting with faculty members inside and outside of instructional settings. As a result, faculty become role models, mentors, and guides for lifelong learning. In addition, effective teaching requires that faculty deliver course material and provide feedback in student-centered ways. Two Engagement Indicators investigate this theme: Student-Faculty Interaction and Effective Teaching Practices. Below are three views of your results alongside those of your comparison groups.

Mean Comparisons

Your first-year students compared with

Score Distributions

Student-Faculty Interaction

Notes: Each box-and-whiskers chart plots the 5th (bottom of lower bar), 25th (bottom of box), 50th (middle line), 75th (top of box), and 95th (top

percentile scores. The dot represents the mean score. Refer to Detailed Statistics for your institution’s sample sizes.

on Indicator Items

Percentage of students who responded that they "Very often" or "Often"…

3a. Talked about

3b. Worked w/faculty on activities other than coursework (committees, student groups, etc.)

3c. Discussed course topics, ideas, or concepts with a faculty member

Percentage responding "Very much" or "Quite a bit" about how much instructors have…

5a.

Experiences with Faculty: Seniors

NSSE 2016 Engagement Indicators

Experiences with Faculty University of Houston-Downtown

Mean

Comparisons

Your seniors compared with

Score Distributions

Notes: Each box-and-whiskers chart plots the 5th (bottom of lower bar), 25th (bottom of box), 50th (middle line), 75th (top of box), and 95th (top

upper

percentile scores. The dot represents the mean score. Refer to Detailed Statistics for your institution’s sample sizes.

Performance on Indicator Items Student-Faculty Interaction

point difference a between your seniors and

Percentage of students who responded that they "Very often" or "Often"…

3a. Talked about career plans with a faculty member

3b. Worked w/faculty on activities other than coursework (committees, student groups, etc.)

3c. Discussed course topics, ideas, or concepts with a faculty member outside of class

3d. Discussed your academic performance with a faculty member

Effective Teaching Practices

Percentage responding "Very much" or "Quite a bit" about how much instructors have…

5a.

NSSE 2016 Engagement Indicators

Campus Environment

University of Houston-Downtown

Campus Environment: First-year students

Students benefit and are more satisfied in supportive settings that cultivate positive relationships among students, faculty, and staff. Two Engagement Indicators investigate this theme: Quality of Interactions and Supportive Environment. Below are three views of your results alongside those of your comparison groups.

Your first-year students compared with

Supportive Environment

Percentage point difference a between your FY

and

Percentage rating their interactions a 6 or 7 (on a scale from 1="Poor" to 7="Excellent") with…

Percentage responding "Very much" or "Quite a bit" about how much the institution emphasized…

14h.

Notes: Refer to your Frequencies and Statistical Comparisons report for full distributions and significance tests. Item numbering corresponds to the survey facsimile included in your Institutional Report and available on the NSSE website. a. Percentage point difference = Institution percentage – Comparison group percentage. Because results are rounded to whole numbers, differences of less than 1 point may or may not display a bar. Small, but nonzero differences may be represented as +0 or -0.

Campus Environment: Seniors

NSSE 2016 Engagement Indicators

Mean Comparisons

Score Distributions

Quality of Interactions Supportive Environment

Percentage rating their interactions a 6 or 7 (on a scale from 1="Poor" to 7="Excellent") with…

13e.

Percentage responding "Very much" or "Quite a bit" about how much the institution emphasized…

14h.

Refer to your Frequencies and Statistical Comparisons report for

This page intentionally left blank.

NSSE 2016 Engagement Indicators

Comparisons with High-Performing Institutions

University of Houston-Downtown

Comparisons with Top 50% and Top 10% Institutions

While NSSE’s policy is not to rank institutions (see nsse.indiana.edu/html/position_policies.cfm), the results below are designed to compare the engagement of your students with those attending two groups of institutions identified by NSSEa for their high average levels of student engagement:

(a) institutions with average scores placing them in the top 50% of all 2015 and 2016 NSSE institutions, and

(b) institutions with average scores placing them in the top 10% of all 2015 and 2016 NSSE institutions.

While the average scores for most institutions are below the mean for the top 50% or top 10%, your institution may show areas of distinction where your average student was as engaged as (or even more engaged than) the typical student at high-performing institutions. A check mark (✓) signifies those comparisons where your average score was at least comparableb to that of the high-performing group. However, the presence of a check mark does not necessarily mean that your institution was a member of that group.

It should be noted that most of the variability in student engagement is within, not between, institutions. Even "high-performing" institutions have students with engagement levels below the average for all institutions.

First-Year Students

Your first-year students compared with

Your seniors compared with UHD

NSSE Top 50% NSSE Top 10% NSSE Top 50%

Top 10%

Notes: Results weighted by institution-reported sex and enrollment status (and institution size for comparison groups); Effect size: Mean difference divided by the pooled standard deviation; *p < .05, **p < .01, ***p < .001 (2-tailed).

a. Precision-weighted means (produced by Hierarchical Linear Modeling) were used to determine the top 50% and top 10% institutions for each Engagement Indicator from all NSSE 2015 and 2016 institutions, separately for first-year and senior students. Using this method, Engagement Indicator scores of institutions with relatively large standard errors were adjusted toward the mean of all students, while those with smaller standard errors received smaller corrections. As a result, schools with less stable data—even those with high average scores—may not be among the top scorers. NSSE does not publish the names of the top 50% and top 10% institutions because of our commitment not to release institutional results and our policy against ranking institutions.

b. Check marks are assigned to comparisons that are either significant and positive, or non-significant with an effect size > -.10.

NSSE 2016 Engagement Indicators

University of Houston-Downtown

Detailed Statistics: First-Year Students

Academic Challenge

Higher-Order Learning

Reflective & Integrative Learning

Learning Strategies

Quantitative Reasoning

Learning with Peers

Collaborative Learning

Discussions with Diverse Others

Experiences with Faculty

NSSE 2016 Engagement Indicators

University of Houston-Downtown

Detailed Statistics: First-Year Students

Student-Faculty Interaction

Effective Teaching Practices

a. Results weighted by institution-reported sex and enrollment status (and institutional size for comparison groups).

b. Standard deviation is a measure of the amount the individual scores deviate from the mean of all the scores in the distribution.

c. Standard error of the mean, used to compute a confidence interval (CI) around the sample mean. For example, the 95% CI (equal to the sample

is the range that is 95% likely to contain the true population mean.

d. A percentile is the point in the distribution of student-level EI scores at or below which a given percentage of EI scores fall.

e. Degrees of freedom used to compute the t -tests. Values vary from the total Ns due to weighting and whether equal variances were assumed.

+/- 1.96 x

f. Statistical significance represents the probability that the difference between the mean of your institution and that of the comparison group occurred by chance.

g. Effect size is the mean difference divided by the pooled standard deviation.

Detailed Statistics: Seniors

Mean statistics

NSSE 2016 Engagement Indicators

Detailed Statisticsa

University of Houston-Downtown

Percentiled scores Comparison results