National Weatherization Assistance Program Impact Evaluation

ORNL/TM-2014/335

National Weatherization Assistance

Program Impact Evaluation – Client

Satisfaction Survey: WAP Service

Delivery from the Client’s Perspective

David Carroll

Jacqueline Berger

Carolyn Miller

Colleen Driscoll

September 2014

DOCUMENT AVAILABILITY

Reports produced after January 1, 1996, are generally available free via US Department of Energy

(DOE) SciTech Connect.

Website http://www.osti.gov/scitech/

Reports produced before January 1, 1996, may be purchased by members of the public from the

following source:

National Technical Information Service

5285 Port Royal Road

Springfield, VA 22161

Telephone 703-605-6000 (1-800-553-6847)

TDD 703-487-4639

Fax 703-605-6900

E-mail [email protected]

Website http://www.ntis.gov/help/ordermethods.aspx

Reports are available to DOE employees, DOE contractors, Energy Technology Data Exchange

representatives, and International Nuclear Information System representatives from the following

source:

Office of Scientific and Technical Information

PO Box 62

Oak Ridge, TN 37831

Telephone 865-576-8401

Fax 865-576-5728

E-mail [email protected]

Website http://www.osti.gov/contact.html

This report was prepared as an account of work sponsored by an

agency of the United States Government. Neither the United States

Government nor any agency thereof, nor any of their employees,

makes any warranty, express or implied, or assumes any legal

liability or responsibility for the accuracy, completeness, or

usefulness of any information, apparatus, product, or process

disclosed, or represents that its use would not infringe privately

owned rights. Reference herein to any specific commercial product,

process, or service by trade name, trademark, manufacturer, or

otherwise, does not necessarily constitute or imply its endorsement,

recommendation, or favoring by the United States Government or

any agency thereof. The views and opinions of authors expressed

herein do not necessarily state or reflect those of the United States

Government or any agency thereof.

ORNL Principal Investigator

Dr. Bruce Tonn

Evaluation Team Task Manager

Jacqueline Berger, Ph.D.

ORNL/TM-2014/335

Environmental Sciences Division

NATIONAL WEATHERIZATION ASSISTANCE PROGRAM

IMPACT EVALUATION – CLIENT SATISFACTION SURVEY: WAP

SERVICE DELIVERY FROM THE CLIENT’S PERSPECTIVE

David Carroll, APPRISE

Jacqueline Berger, Ph.D., APPRISE

Carolyn Miller, Carolyn Miller Consulting

Colleen Driscoll, APPRISE

Date Published: September 2014

Prepared by

OAK RIDGE NATIONAL LABORATORY

Oak Ridge, Tennessee 37831-6283

managed by

UT-BATTELLE, LLC

for the

US DEPARTMENT OF ENERGY

under contract DE-AC05-00OR22725

iii

CONTENTS

Page

LIST OF TABLES ........................................................................................................................................ v

ACRONYMS AND ABBREVIATIONS ................................................................................................... vii

ACKNOWLEDGEMENTS ......................................................................................................................... ix

EXECUTIVE SUMMARY ......................................................................................................................... xi

1. INTRODUCTION ................................................................................................................................ 1

1.1 NATIONAL WEATHERIZATION ASSISTANCE PROGRAM EVALUATION

OVERVIEW ............................................................................................................................... 1

1.2 SATISFACTION SURVEY STUDY OVERVIEW ................................................................... 2

1.3 ORGANIZATION OF THE CLIENT SATISFACTION SURVEY REPORT .......................... 2

2. SURVEY DESIGN AND IMPLEMENTATION ................................................................................ 3

2.1 SATISFACTION SURVEY METHODOLOGY ....................................................................... 3

2.2 ANALYSIS FRAMEWORK ...................................................................................................... 4

2.3 DATA COLLECTION STATISTICS ........................................................................................ 4

3. SATISFACTION WITH THE WEATHERIZATION PROCESS ....................................................... 7

3.1 APPLICATION AND INTAKE ................................................................................................. 7

3.2 AUDIT ...................................................................................................................................... 11

3.3 WEATHERIZATION SERVICES ........................................................................................... 12

3.4 FINAL INSPECTION .............................................................................................................. 15

4. CLIENT EDUCATION ...................................................................................................................... 19

4.1 ENERGY EDUCATION .......................................................................................................... 20

4.2 HEALTH AND SAFETY EDUCATION ................................................................................. 25

5. OVERALL PROGRAM SATISFACTION ........................................................................................ 33

5.1 AREAS FOR IMPROVEMENT .............................................................................................. 33

5.2 OVERALL PROGRAM SATISFACTION .............................................................................. 37

5.3 SATISFACTION BENCHMARKS ......................................................................................... 39

5.4 OVERALL PROGRAM BENEFITS ........................................................................................ 40

LIST OF TABLES

Table Page

Table 1 Overall Satisfaction with Key Components of the Program ......................................................... xvi

Table 2 Ease of Dealing with Different Aspects of the Program ............................................................... xvi

Table 3 Timeliness of Various Components of the Program ..................................................................... xvi

Table 3.1 Awareness of Local Weatherization Program .............................................................................. 8

Table 3.2 How Learned about Local Weatherization Program ..................................................................... 8

Table 3.3 Time Since First Request for Weatherization ............................................................................... 9

Table 3.4 Satisfaction with Time Between Request and Receipt of Services............................................... 9

Table 3.5 Ease of Weatherization Request ................................................................................................. 10

Table 3.6 Reasons for Weatherization Application .................................................................................... 10

Table 3.7 Actions Taken by the Household Prior to Weatherization ......................................................... 11

Table 3.8 Ease of Initial Audit Scheduling ................................................................................................. 12

Table 3.9 Timeliness of Auditors ................................................................................................................ 12

Table 3.10 Courteousness of Auditors ........................................................................................................ 12

Table 3.11 Ease of Weatherization Installation Scheduling ....................................................................... 13

Table 3.12 Timeliness of Weatherization Crew .......................................................................................... 14

Table 3.13 Courteousness of Weatherization Crew .................................................................................... 14

Table 3.14 Carefulness of Weatherization Crew ........................................................................................ 14

Table 3.15 Cleanliness of Crew .................................................................................................................. 15

Table 3.16 Satisfaction with Final Home Condition ................................................................................... 15

Table 3.17 Final Inspection Received ......................................................................................................... 16

Table 3.18 Ease of Final Inspection Scheduling ......................................................................................... 16

Table 3.19 Timeliness of Final Inspector .................................................................................................... 17

Table 3.20 Courteousness of Final Inspector .............................................................................................. 17

Table 4.1 Information Provided on Ways to Save Energy .......................................................................... 20

Table 4.2 Time Spent Talking About Ways to Save Energy ...................................................................... 21

Table 4.3 Understanding of Verbal Information Provided ......................................................................... 21

Table 4.4 Staff Provided Educational Materials about Saving Energy ....................................................... 22

Table 4.5 Respondent Time Spent Reading or Reviewing Materials about Saving Energy ....................... 22

Table 4.6 Respondent Understanding of Materials about Saving Energy .................................................. 23

Table 4.7 Usefulness of Materials about Saving Energy ............................................................................ 23

Table 4.8 Useful Features of Materials about Saving Energy .................................................................... 24

Table 4.9 Ways to Improve Materials about Saving Energy ...................................................................... 24

Table 4.10 Overall Satisfaction with Energy Saving Information Provided ............................................... 25

Table 4.11 Staff Knowledge about Health of Household Members ........................................................... 25

Table 4.12 Assistance for Household Members in Need of Health Care .................................................. 26

Table 4.13 Referrals to Social Service Programs ....................................................................................... 27

Table 4.14 Staff Provided Information on Ways to Improve Health and Safety in the Home ................... 27

Table 4.15 Staff Time Spent Talking About Improving Health and Safety ............................................... 28

Table 4.16 Respondent Understanding of Information on Improving Health and Safety .......................... 28

Table 4.17 Health and Safety Materials Provided ...................................................................................... 29

Table 4.18 Time Reviewed Health and Safety Materials ........................................................................... 29

Table 4.19 Respondent Understanding of Materials Provided About Improving Health and Safety ......... 30

Table 4.20 Usefulness of Materials about Improving Health and Safety ................................................... 30

Table 4.21 Useful Features of the Materials on Improving Health and Safety ........................................... 31

Table 4.22 How Materials on Health and Safety could be Improved ......................................................... 31

Table 4.23 Satisfaction with Health and Safety Information ...................................................................... 32

Table 4.24 Potential Improvements to Health and Safety Information....................................................... 32

Table 5.1 Problems with Weatherization Services Received ...................................................................... 33

Table 5.2 Complaint Filed about Weatherization Services ......................................................................... 34

Table 5.3 Household Perceived Need for Additional Measures ................................................................. 34

Table 5.4 Additional Weatherization Measures .......................................................................................... 35

Table 5.5 Staff Checked for Major Repairs ................................................................................................ 36

Table 5.6 Suggestions for Program Improvement ...................................................................................... 36

Table 5.7 Overall Satisfaction with Work Done ......................................................................................... 37

Table 5.8 Overall Satisfaction with New Equipment Installed ................................................................... 37

Table 5.9 Satisfaction with Energy Savings after Weatherization .............................................................. 38

Table 5.10 Overall Satisfaction with the Weatherization Program ............................................................. 38

Table 5.11 Overall Satisfaction with Key Components of the Program ..................................................... 39

Table 5.12 Ease of Dealing with Different Aspects of the Program ........................................................... 40

Table 5.13 Timeliness of Various Components of the Program ................................................................. 40

Table 5.14 Benefits of Program Participation ............................................................................................. 40

Table 5.15 Word of Mouth Referral ........................................................................................................... 41

Table 5.16 Impact of Services on Likelihood of Moving ........................................................................... 41

Table 5.17 Likeliness Household Will Move in Next 12 months ............................................................... 42

vii

ACRONYMS AND ABBREVIATIONS

CATI

Air Conditioning

Computer Assisted Telephone Interview

CDD

Cooling Degree Days

CFL

Compact Fluorescent Light

CFM50

Cubic Feet per Minute @ 50 pascals

CFR

Code of Federal Regulations

Carbon Monoxide

DOE

Department of Energy

ECM

Energy Conservation Measure

ECW

Energy Center of Wisconsin

HDD

Heating Degree Days

KWH

LIHEAP

Kilowatt Hour

Low Income Home Energy Assistance Program

MMBTU

Million British Thermal Units

ORNL

OWIP

Oak Ridge National Laboratory

Office of Weatherization and Intergovernmental Programs

Program Year

Prescription Medications

SOW

Scope of Work

SSE

Steady State Efficiency

Therm

WAP

Weatherization Assistance Program

ACKNOWLEDGEMENTS

The work presented in this report was funded by the U.S. Department of Energy’s (DOE) Office of

Weatherization and Intergovernmental Programs (OWIP).

This report summarizes the findings from the Satisfaction Survey of the Weatherization Assistance

Program (WAP) Evaluation. This survey was the second of three surveys conducted with households that

received WAP services; it collected information from 665 clients regarding their satisfaction with the

services delivered by the program.

The original design for the survey was developed by staff from the Oak Ridge National Laboratory

(ORNL) as one component of the National Evaluation of the Weatherization Assistance Program.

(National Evaluation of the Weatherization Assistance Program: Preliminary Evaluation Plan for

Program 2006 – ORNL/CON-498). As part of the evaluation plan development, the design team

consulted with and received feedback from the Network Planning Committee, 41 individuals from the

weatherization network. Erin Rose of ORNL developed the final version of the survey instrument.

ORNL contracted with the research team of APPRISE Incorporated, the Energy Center of Wisconsin,

Michael Blasnik and Associates, and Dalhoff Associates LLC to conduct the National Evaluation.

APPRISE took primary responsibility for the design and implementation of this study. The members of

the Evaluation Team who contributed to the design and implementation of this survey included:

APPRISE Incorporated The Energy Center of Wisconsin

Jackie Berger Claire Cowan

David Carroll LaShanta Goodwin

Colleen Driscoll Jaimie Rule

Jennifer Frenett

Ferit Ucar Carolyn Miller Consulting

Anne Worth Carolyn Miller

I.C. International prepared the CATI survey instrument, conducted the telephone interviews, and

delivered a data file for this study.

This study could only be completed with the cooperation and contributions of the 51 grantees and 220

subgrantees who furnished lists of clients. In addition, the study depended on the willingness of 665

clients to share their experiences with the evaluation team.

Finally, we would like to acknowledge the assistance and guidance of the Oak Ridge National

Laboratory, Department of Energy Project Officers, and Department of Energy Headquarters Staff.

Jackie Berger

David Carroll

EXECUTIVE SUMMARY

The purpose of this report is to disseminate the findings from an analysis of the WAP Client Satisfaction

Survey. The Satisfaction Survey builds on the Baseline Occupant Survey. That survey was conducted in

2011 with two groups of households; one group that applied for WAP services in program year (PY) 2011

(i.e., treatment group) and a second group that had been served by WAP one year earlier in PY 2010 (i.e.,

comparison group). The Client Satisfaction Survey was conducted with the Baseline Occupant Survey

respondents who applied for WAP services in PY 2011 and were served by the program. The primary

focus of this survey was to document clients’ perceptions of the weatherization program and their

satisfaction with the services that they received.

Background

The U.S. Department of Energy’s (DOE) Weatherization Assistance Program was created by Congress in

1976 under Title IV of the Energy Conservation and Production Act. The purpose and scope of the

Program as currently stated in the Code of Federal Regulations (CFR) 10 CFR 440.1 is “to increase the

energy efficiency of dwellings owned or occupied by low-income persons, reduce their total residential

energy expenditures, and improve their health and safety, especially low-income persons who are

particularly vulnerable such as the elderly, persons with disabilities, families with children, high

residential energy users, and households with high energy burden.” (Code of Federal Regulations, 2011)

At the request of DOE, Oak Ridge National Laboratory (ORNL) developed a comprehensive plan for a

national evaluation of WAP that was published in 2007. DOE furnished funding to ORNL in 2009 for a

national evaluation for Program Years 2007 and 2008, with a particular emphasis on PY 2008. The Scope

of Work (SOW) for the evaluation includes an Impact Assessment, a Process Assessment, Special

Technical Studies, and a Synthesis Report. The Occupant Surveys contribute to both the Impact and

Process Assessments.

Occupant Study Overview

The Client Satisfaction Survey is one of three surveys conducted with the WAP clients. The Baseline

Survey documents status and needs of clients prior to weatherization. The Client Satisfaction Survey

collects information on client perceptions of WAP service delivery. The follow-up Occupant Survey

directly assesses household budget and energy behaviors pre- to post-weatherization. This is one of

several reports that will be completed using findings from the Occupant Surveys. The full set of reports

includes:

 Client Satisfaction Survey Report: WAP Service Delivery from the Client’s Perspective (Carroll,

et al. 2014)

 Baseline Occupant Survey Report: Assessment of Client Status and Needs (Carroll, et al. 2014)

 Survey of Recipients of Weatherization Assistance Program Services: Assessment of Household

Budget and Energy Behaviors Pre- to Post-Weatherization (Tonn, et al. 2015)

 Health and Household Related Benefits Attributable to the WAP Program (Tonn, et al. 2014)

 Exploratory Review of State, Agency and Client Experiences with Deferred Services Under the

Weatherization Assistance Program (Rose, et al. 2014)

The findings from the Client Satisfaction Survey will contribute to the development of WAP Program

Process Evaluation Report.

xii

Satisfaction Survey Overview and Methodology

The overarching objective of the Satisfaction Survey is to measure client perceptions about the delivery of

weatherization services and the value of the services that they received. The survey topics include:

 Satisfaction with WAP Program Components

 Client Education in the Context of WAP Service Delivery

 Overall WAP Program Satisfaction

In the Baseline Occupant Survey, interviews were completed with 1,094 Treatment Group clients and 803

Comparison Group clients. The Client Satisfaction Survey was conducted with Treatment Group clients.

The survey procedures included:

 Client Sample – All Treatment Group respondents were eligible for the survey.

 Round 1 – Three months after the Baseline Survey, Treatment Group respondents were contacted

and interviewed if they had received weatherization services. (N=418)

 Round 2 – Nine months after the Baseline Survey, Treatment Group respondents who had not

received services after three months were contacted and interviewed if they had received

weatherization services by that time. (N=247)

This report focuses on the 665 Treatment Group clients who completed the Satisfaction Survey.

Satisfaction with the Weatherization Process

When a client receives weatherization services, there are usually four steps in the service delivery

process, including:

 Application and Intake – Verification that the household is eligible for the program.

 Audit – Survey of the home to determine which measures would be cost-effective.

 Weatherization Service Delivery – Installation of cost-effective measures.

 Inspection – Verification that measures were installed properly.

In general, the findings from this section show that clients were very satisfied with all aspects of the

program service delivery. Weatherization auditors, crews, and inspectors were on time, courteous, and

professional. The weatherization crews were careful in their approach to the work and were sure to clean

up both inside and outside the home after service delivery.

With respect to the application and intake process, key findings include:

 Application to Service Delivery Interval – The interval between application and service delivery

varied considerably for clients; 56 percent of clients were served within one year of application,

but 18 percent had to wait two years or more.

 Satisfaction with Interval – Despite the long time interval for some households, clients were

satisfied with the process; 84 percent were either satisfied or very satisfied.

 Ease of Weatherization Request – Agencies made the process easy for clients; 87 percent reported

that requesting weatherization services was either easy or very easy.

 Reasons for Weatherization Application – Most clients reported that they applied for

weatherization to reduce energy bills (78 percent) or make their home more comfortable (67

xiii

percent). Some clients reported that they wanted to support environmental efforts by conserving

energy (37 percent).

 Actions Taken Prior to Weatherization – Most clients took actions on their own before receiving

weatherization services; 34 percent purchased draft stoppers, 33 percent installed foam tape to

stop drafts, and 10 percent repaired or replaced broken windows. Only 19 percent said that they

had not done anything.



With respect to the energy audit process, key findings include:

 Ease of Scheduling – Almost all clients (94 percent) said that it was easy or very easy to schedule

the home energy audit.

 Timeliness of Auditor – Almost all clients (95 percent) said that the auditor was on time. Only 1

percent of clients reported that the auditor was more than an hour late.

 Courteousness of Auditor – Almost all clients (98 percent) said that the auditor was courteous or

very courteous.

With respect to the weatherization measure installation process, key findings include:

 Ease of Scheduling – Almost all clients (93 percent) said that it was easy or very easy to schedule

a time for the weatherization crew to come to their home.

 Timeliness of Weatherization Crew – Almost all clients (93 percent) said that the weatherization

crew was on time. Only 2 percent of clients reported that the crew was more than an hour late.

 Courteousness of Weatherization Crew – Almost all clients (96 percent) said that the

weatherization crew was courteous or very courteous.

 Carefulness of Weatherization Crew – Almost all clients (93 percent) said that the weatherization

crew was careful or very careful. Only 1 percent said that the crew was very careless.

 Cleanliness of Weatherization Crew – Most clients (89 percent) said that the weatherization crew

left their home clean. Only 3 percent reported that the crew left their home dirty.

 Final Condition of Home - Almost all clients (93 percent) said that they were satisfied or very

satisfied with the final condition of their home.

xiv

With respect to the final inspection process, key findings include:

 Ease of Scheduling – Almost all clients (97 percent) said that it was easy or very easy to schedule

the final inspection.

 Timeliness of Auditor – Almost all clients (97 percent) said the inspector was on time. Less than

1 percent of clients reported that the inspector was more than an hour late.

 Courteousness of Auditor – Almost all clients (97 percent) said that the inspector was courteous

or very courteous.

The weatherization process - conducting audits, delivering weatherization services, and conducting final

inspections - can be challenging to schedule and complicated to complete effectively. However, these

high grades (90 percent or above on almost all measures) speak well of the systems that the

weatherization network has put in place to deliver these services.

Client Education

The Satisfaction Survey asked clients to describe the extent to which the program furnished information

on energy efficiency and health and safety, and to report on their satisfaction with those components of

the program.

The survey found that most, but not all, clients received energy education services. When they received

services, the clients generally reported that the information was useful. However, compared to the

investment in installation of energy efficiency measures and resolution of health and safety problems, the

investment in direct energy education was modest and did not appear to have a major impact on the

client’s weatherization experience. Most clients reported that they had less than 30 minutes of energy

education. Some key findings with respect to energy education include:

 Incidence – 69 percent of clients reported that they received some information from

weatherization staff on ways to save energy in their home.

 Intensity – 22 percent of clients reported that staff spent 30 minutes or more talking to them about

ways to save energy. The median amount of time was 14 minutes.

 Quality – 96 percent of clients who received energy education said that they understood the

information either well or very well.

 Materials – 50 percent of clients reported that they received some educational materials on saving

energy.

 Material Review – 70 percent of clients who received energy education materials spent 15

minutes or more reviewing the materials.

 Material Quality – 96 percent of clients who received energy education materials said that they

understood the materials either well or very well.

 Overall Satisfaction – 94 percent of the clients who received energy education reported that they

were either satisfied or very satisfied with the information.

The survey found that less than half of the weatherization clients reported that they had significant

interactions with the weatherization program staff in terms of health and safety issues. Among those

clients who had in-depth discussions about health and safety issues, it is clear that the clients understood

some important messages about how to protect their health and the health of their families, both in terms

of the safety of energy efficiency equipment and in terms of the impact of other factors on the indoor air

quality in their homes. However, the service delivery process did not focus as much on health and safety

as it did on energy efficiency. Some key findings with respect to health and safety education include:

 Health Status – 50 percent of clients reported that they were either asked about health issues by

weatherization staff or they volunteered information on health issues to the weatherization staff.

 Unmet Health Needs – Only 5 percent of clients who reported on health status indicated that there

was at least one household member with unmet healthcare needs.

 Referrals – 11 percent of clients reported that weatherization staff referred them to social service

programs; however, most of the referrals were to cash or in-kind assistance programs rather than

health programs.

 Health and Safety Education – 40 percent of clients reported that weatherization staff furnished

information on ways to improve health and safety in their home.

 Intensity – Clients who received information talked to weatherization staff for an average of 20

minutes.

 Quality – 98 percent of clients who received health and safety education said that they understood

the information either well or very well.

 Materials – 24 percent of clients reported that they received some educational materials on health

and safety.

 Material Review – Clients who received health and safety education materials spent an average of

15 minutes reviewing the materials.

 Material Quality – 96 percent of clients who received health and safety education materials said

that they understood the materials either well or very well.

 Overall Satisfaction – 95 percent of the clients who received health and safety education reported

that they were either satisfied or very satisfied with the information.

These findings might suggest that the program could do more client education, particularly for health and

safety issues. However, the survey only tells us about the experience of clients, not the reason for those

experiences. With respect to energy efficiency education, many service delivery professionals suggest that

they can be more useful by engaging clients in the weatherization process, rather than by talking about

savings practices independent of the installed measures. In terms of health and safety education, service

delivery staff may only focus on these issues for homes and households where there clearly is a problem.

In both the Baseline Survey and in the Satisfaction Survey, most clients did not report any major health

issues or safety concerns. One possible interpretation of the results is that service delivery staff are

responding to education opportunities, rather than conducting education with every client.

Overall, many clients receive energy education and some clients receive health and safety education.

While most clients seem satisfied with the services, it is not clear that they see this as a major benefit of

the program. And, it does not seem like there has been a significant investment in terms of the time spent

on education. The findings from this survey suggest that client education in the program is treated as a

service that supports the weatherization process, rather than being a major focus of the program. If

policymakers perceive that either energy education or health and safety education can deliver major

benefits to low-income households, it seems that some change in either the delivery of those services or in

the program guidance would be required to achieve that level of benefit.

Overall Program Satisfaction

The Satisfaction Survey shows that the Weatherization Assistance Program gets high marks from clients;

over 90 percent of clients are satisfied or very satisfied with most aspects of service delivery. But, it is

useful to consider where clients give the program the highest rating - "very satisfied" - compared to those

program elements where they are more likely to say that they are "satisfied." This offers some insight on

where there is some room for improvement. Table 1 shows that the program gets consistently high marks

xvi

for the final conditions inside and outside the home, the work performed, the new equipment, and the

overall program. The areas where clients were less likely to report that they are very satisfied are

information on health and safety measures, information on saving energy, and the length of time between

the request for weatherization and the time when the home was weatherized. Clients gave a lower rating

to the level of energy savings in the home, but that may be partially because, for many clients, not enough

time had passed to observe the level of energy savings.

Table 1 Overall Satisfaction with Key Components of the Program

How satisfied are you with the ________?

Number of Respondents

Very Satisfied (665)

Satisfied (665)

Length of time between request to have home weatherized and when

it was weatherized

44%

39%

Final condition of the inside of the home

58%

35%

Final condition of the outside of the home

59%

33%

Work performed in the home

60%

33%

New equipment installed in the home

61%

30%

Energy savings achieved after having the home weatherized

35%

22%

Information on saving energy

37%

28%

Information on improving health and safety

22%

16%

Overall Weatherization Program

63%

31%

The survey also showed that the program got high ratings in terms of client service. Table 2 shows that

almost all clients found that it was easy or very easy to schedule work and Table 3 shows that almost all

of the professionals who came to the home were either early or on time.

Table 2 Ease of Dealing with Different Aspects of the Program

How easy was it to ________?

Respondents

Very Easy

Easy

Request that the home be weatherized

657

40%

47%

Schedule the initial audit

660

49%

45%

Schedule the weatherization crew to come to home

661

53%

40%

Schedule the final inspection

502

62%

35%

Table 3 Timeliness of Various Components of the Program

How timely was/were ________?

Respondents

Percent who said

‘early or on time’

The people who did the initial audit of the home

651

95%

The weatherization crew

658

93%

The people who did the final inspection of the home

501

97%

xvii

The Satisfaction Survey showed that clients clearly understand the benefits for the program. Clients

perceive that:

 Their homes use less energy

 Their homes are more comfortable

 Their homes have better energy equipment

 They are now less likely to have to move from their home

The survey also showed that clients tell other low-income households about the program and that some of

those households get weatherized. It is hard to have a better testimonial regarding the value of the

program to low-income households and their satisfaction with program services than the direct and

obviously compelling recommendation to their friends and neighbors.

1. INTRODUCTION

The purpose of this report is to disseminate the findings from an analysis of the WAP Client Satisfaction

Survey. It is one of three surveys conducted with households that received WAP services. The Baseline

Survey documents status and needs of clients prior to weatherization. The Satisfaction Survey collects

information on client perceptions of WAP service delivery. The follow-up Occupant Survey directly

assesses household budget and energy behaviors pre- to post-weatherization.

This is one of several reports that will be completed using findings from the Occupant Surveys. The full

set of reports includes:

 Client Satisfaction Survey Report: WAP Service Delivery from the Client’s Perspective (Carroll, et

al. 2014)

 Baseline Occupant Survey Report: Assessment of Client Status and Needs (Carroll, et al. 2014)

 Survey of Recipients of Weatherization Assistance Program Services: Assessment of Household

Budget and Energy Behaviors Pre- to Post-Weatherization (Tonn, et al. 2015)

 Health and Household Related Benefits Attributable to the WAP Program (Tonn, et al. 2014)

 Exploratory Review of State, Agency and Client Experiences with Deferred Services Under the

Weatherization Assistance Program (Rose, et al. 2014)

The Client Satisfaction Survey Report and the Deferral Report contribute to the WAP Program Process

Report.

1.1 NATIONAL WEATHERIZATION ASSISTANCE PROGRAM EVALUATION

OVERVIEW

The U.S. Department of Energy’s (DOE) Weatherization Assistance Program was created by Congress in

1976 under Title IV of the Energy Conservation and Production Act. The purpose and scope of the

Program as currently stated in the Code of Federal Regulations (CFR) 10CRF 440.1 is “to increase the

energy efficiency of dwellings owned or occupied by low-income persons, reduce their total residential

energy expenditures, and improve their health and safety, especially low-income persons who are

particularly vulnerable such as the elderly, persons with disabilities, families with children, high

residential energy users, and households with high energy burden.” (Code of Federal Regulations, 2011)

At the request of DOE, Oak Ridge National Laboratory (ORNL) developed a comprehensive plan for a

national evaluation of WAP that was published in 2007. (National Evaluation of the Weatherization

Assistance Program: Preliminary Evaluation Plan for Program Year 2006, February 2007) DOE furnished

funding to ORNL in 2009 for a national evaluation for Program Years 2007 and 2008, with a particular

emphasis on PY 2008. ORNL subcontracted evaluation research to APPRISE Incorporated and its

partners (the Energy Center of Wisconsin, Michael Blasnik and Associates, and Dalhoff Associates LLC).

The Scope of Work (SOW) for the evaluation includes the following components.

 Impact Assessment – Characterization of the weatherization network and the households that are

income-eligible for WAP. Measurement and monetization of the energy and nonenergy impacts

of the program. Assessment of the factors associated with higher levels of energy savings, cost

savings, and cost-effectiveness.

 Process Assessment – Direct observation of how the weatherization network delivers services and

assessment of how service delivery compares to national standards. Documentation of how

weatherization staff and clients perceive service delivery.

 Special Technical Studies – Examination of the performance of the program with respect to

technical issues such as air sealing, duct sealing, furnace efficiency, and refrigerators.

 Synthesis Study – Synthesis of the findings from this evaluation into a comprehensive assessment

of the success of the program in meeting its goals and identification of key areas for program

enhancement.

The Occupant Surveys contribute to both the Impact Assessment and the Process Assessment. WAP

program non-energy benefits will be measured for the Impact Assessment by comparing the pre-

weatherization status of WAP clients in the Baseline Survey to the post-weatherization status of WAP

clients in the Follow-Up Survey. WAP client perceptions will be measured for the Process Assessment

through the Client Satisfaction Survey. WAP procedures with respect to deferral of clients will be

measured for the Process Assessment through the Follow-Up survey with deferred clients.

1.2 SATISFACTION SURVEY STUDY OVERVIEW

The overarching objective of the Satisfaction Survey is to measure client perceptions about the delivery of

weatherization services and the value of the services that they received. The survey topics include:

 Satisfaction with WAP Program Components

 Client Education in the Context of WAP Service Delivery

 Overall WAP Program Satisfaction

The Satisfaction Survey was conducted with the respondents to the Baseline Occupant Survey who

applied for WAP in PY 2011. The sample for the Baseline Occupant Survey included households that had

applied for WAP services, were determined to be income-eligible for the program, and were scheduled

for a home energy audit during PY 2011. The households were interviewed prior to receiving their home

energy audit. The first round of the Satisfaction Survey was conducted three months after the Baseline

interview. If the respondent had received services, they were interviewed. If a respondent had not yet

received services, their interview was deferred. The second round of the Satisfaction Survey was

conducted nine months after the Baseline Interview. If a respondent had received services, they were

interviewed. If a respondent had not received services, they were asked about their perceptions of why

WAP services were not delivered and were included in the sample for the Deferral Survey.

1.3 ORGANIZATION OF THE CLIENT SATISFACTION SURVEY REPORT

This report consists of five sections, including:

 Section 1 - Introduction: Furnishes an overview of the Weatherization Assistance Program

Evaluation and the purpose of the Client Satisfaction Survey.

 Section 2 – Survey Design and Implementation: Outlines the Satisfaction Survey sampling and

data collection procedures.

 Section 3 - Satisfaction with WAP Program Components: Examines information on WAP clients’

initial program awareness and experience with different phases of the program.

 Section 4 – Client Education in the Context of WAP Service Delivery: Analyzes what

information clients retained on energy saving behaviors, and health and safety practices from the

WAP visits.

 Section 5 – Overall WAP Program Satisfaction: Provides statistics of clients’ overall satisfaction

with the program, their perception of program benefits, and their suggestions for improvement.

These findings are for a representative national sample of clients served in PY 2011.

2. SURVEY DESIGN AND IMPLEMENTATION

The Client Satisfaction Survey is one of three surveys that are planned to be conducted with the selected

WAP clients. The Baseline Survey documented status and needs of clients prior to weatherization. The

Satisfaction Survey collected information on client perceptions of the WAP program and WAP service

delivery. The Follow-Up Survey will measure how the status and needs of clients have changed one year

after receiving WAP services.

2.1 SATISFACTION SURVEY METHODOLOGY

The initial survey procedures for the Baseline Occupant Survey included the following:

 Agency Sample – A sample of 220 service delivery agencies was selected to represent the 905

WAP service agencies nationwide.

 Treatment Group Sample – Each agency was asked to furnish a list of clients who was income-

qualified for the program and that was scheduled for a WAP audit.

 Comparison Group Sample – Each agency was asked to furnish a list of clients who received

WAP services one year earlier.

 Interviews – Sampled treatment and comparison group clients were contacted and interviewed

using a survey instrument designed by the ORNL Evaluation Team.

Interviews were completed with 1,094 treatment group clients and 803 comparison group clients.

The survey procedures for the Client Satisfaction Survey included the following:

 Client Sample – All Treatment Group survey respondents (n=1,094) were eligible for the

Satisfaction Survey.

 Round 1 – Three months after the completion of the Baseline Survey field period, Treatment

Group survey respondents were contacted and screened using a survey instrument designed by

the ORNL Evaluation Team. If the survey respondent had received weatherization services, they

were interviewed. If they had not received weatherization services, their interview was deferred.

418 Baseline Survey respondents were interviewed in Round 1 of the Client Satisfaction Survey.

 Round 2 – Nine months after the completion of the Baseline Survey field period, deferred

Treatment Group survey respondents were contacted and screened using a survey instrument

services, they were interviewed. 247 Baseline Survey respondents were interviewed in Round 2

of the Client Satisfaction Survey. If the survey respondent had not received weatherization

services, they were asked questions related to service deferral.

 Survey Statistics – In total, 665 of the Treatment Group clients received WAP services, continued

to live in the weatherized housing unit, and could be contacted for follow-up interviews. Those

Baseline Survey respondents completed the Satisfaction Survey interview. Among the 1,094

treatment group clients surveyed prior to receiving a WAP audit, 290 reported that they had not

received WAP services and 139 could no longer be reached by telephone.

This report will focus on the 665 Treatment Group clients who completed the Client Satisfaction Survey

interview.

2.2 ANALYSIS FRAMEWORK

The primary purpose of the Occupant Surveys is to furnish information on the energy status and needs of

applicant households and to assess the extent to which the WAP program is able to address those needs.

The primary analysis sample for the study is the treatment group; households that have applied for the

WAP program, have been determined to be income qualified for the program, and were scheduled for a

WAP home energy audit. This study is designed to:

 Develop an understanding of their energy status and needs prior to any significant contact with

the program,

 Determine whether the WAP program was able to deliver services to them,

 Assess their perceptions of the effectiveness of the WAP program in meeting their needs, and

 Measure the change in status and needs of the household after having received WAP services.

The three surveys contribute to this analysis in the following way.

 Baseline Survey – The Baseline Survey was conducted with WAP clients prior to their home

energy audit. While clients may have had some engagement with the WAP program by applying

for program services, these interviews represent, as much as possible, the needs of the applicant

households prior to receiving services from the WAP program.

 Client Satisfaction Survey – The Client Satisfaction Survey was conducted with households that

received WAP services. It collected information on client perceptions of the WAP program and

WAP service delivery.

 Follow-Up Survey – The Follow-Up Survey is planned to be conducted at the end of the winter of

2012-2013. The survey will be conducted with all treatment group households, including those

that were served and those that were deferred. It shows how the status of households changed

after receipt of WAP program services.

One important component of the research design for the Baseline and Follow-Up Surveys is that a

comparison group of WAP clients was interviewed. The treatment group households were scheduled to

receive WAP services during PY 2011. A comparison group of households that received services during

PY 2010 also was sampled and interviewed. The primary purpose of the comparison group is to furnish a

“difference of differences” analysis. In such a design, the gross program impact is the change in the

treatment group status. But, the net program impact is determined by measuring the status change for a

comparison group and netting out any change for the comparison group against the change for the

treatment group to control for other unobserved factors.

Since the primary focus of the Satisfaction Survey is to ascertain client perceptions of weatherization

services and the weatherization program a net difference analysis is not needed. Therefore, the

Satisfaction Survey did not include the comparison group households.

2.3 DATA COLLECTION STATISTICS

The Client Satisfaction Survey used sample development and interviewing procedures that were designed

to achieve the highest possible response rate. The Satisfaction Survey Sample came from the treatment

group households that responded to the Baseline Occupant Survey.

The original agency contacts for the Occupant Surveys were made by Energy Center of Wisconsin

(ECW) case managers who have been working with service delivery agencies since 2010 to facilitate data

collection for the overall evaluation project. For each sampled service delivery agency:

 Advance Mailing – The ECW case manager mailed information to the agency contact explaining

the purpose of the data collection and the procedures for selecting and delivering client lists.

 Agency Phone Contact – The ECW case manager contacted the agency contact to discuss the

data collection schedule and to clarify the study procedures.

 Agency Follow-Up – The ECW case manager conducted regular follow-up with the agency

contact to facilitate the development and delivery of the clients lists.

Of the 220 sampled agencies, 204 (93%) furnished client lists. The survey was successful in getting

completed interviews from 203 of the 204 agencies that furnished client lists.

The Baseline Occupant Survey implementation procedures included:

 Advance Mailing – Sampled households were sent a letter that alerted them that they would be

contacted by Braun Research, described the purpose of the survey, furnished a call-in 800 number

that would allow them to do the survey at their convenience, and notified them that they would

receive a $20 incentive for completing the interview.

 Phone Contact – Braun Research made at least 12 call attempts to each household at different

times of the day and on different days of the week. On the first call attempt, and every third call

attempt thereafter, interviewers left messages on answering machines.

 Refusal Conversion – If a potential respondent refused to conduct the interview, they were called

back by a refusal conversion team that attempted to address any concerns about completing the

interview.

 Spanish Language Interviews – When the telephone center encountered Hispanic households with

a language barrier, an interviewer re-contacted the households and conducted the interview in

Spanish.

Interviewers completed 1,094 Baseline Occupant Survey interviews with Treatment Group households

with an estimated response rate of 73%.

The Treatment Group respondents were eligible for the Client Satisfaction Survey. The survey

implementation procedures included:

 Advance Mailing – Sampled households were sent a letter that alerted them that they would be

contacted by IC International, described the purpose of the survey, furnished a call-in 800 number

that would allow them to do the survey at their convenience, and notified them that they would

receive a $10 incentive for completing the interview.

 Phone Contact – IC International made at least 12 call attempts to each household at different

times of the day and on different days of the week. On the first call attempt, and every third call

attempt thereafter, interviewers left messages on answering machines.

 Refusal Conversion – If a potential respondent refused to conduct the interview, they were called

back by a refusal conversion team that attempted to address any concerns about completing the

interview.

 Spanish Language Interviews – When the telephone center encountered Hispanic households with

a language barrier, an APPRISE interviewer re-contacted the households and conducted the

interview in Spanish.

Interviewers completed 665 Client Satisfaction Survey interviews with an estimated response rate of 87%.

3. SATISFACTION WITH THE WEATHERIZATION PROCESS

This section of the report provides findings on respondents' initial awareness of the program and their

program experiences. The section is divided into four sub-sections that look at each step in the service

delivery process, including:

 Application and Intake

 Audit

 Weatherization Service Delivery

 Final inspection

In general, the findings from this section show that clients were very satisfied with all aspects of the

program service delivery. Weatherization auditors, crews, and inspectors were on time, courteous, and

professional. The weatherization crews were careful in their approach to the work and cleaned up both

inside and outside the home after service delivery.

3.1 APPLICATION AND INTAKE

The first part of the Client Satisfaction Survey asked respondents to report on their experiences with

application and intake, including:

 How and when the respondent first learned of the program

 How long it took between request for and receipt of services

 Satisfaction with timeliness of services

 Ease of request for services

 Why the respondent applied

 Weatherization actions taken prior to program participation

In reviewing the responses furnished by clients in this section, it is important to consider two important

factors associated with how clients experience program application and intake.

 First, a client’s application and intake experience is likely to vary depending on how they were

referred to the program; the experience for a direct Low Income Home Energy Assistance

Program (LIHEAP) referral may be quite different than from the experience for a household

that learned of the WAP program from other sources, had to apply for the program, and had to

furnish information to qualify for the program.

 Second, the experiences of clients during the ARRA funding period of the program may be

different from the experiences of clients who applied for and were served prior to the ARRA

funding period.

From those perspectives, the Client Satisfaction Survey furnishes useful information about the application

and intake experiences of clients during the ARRA period, but may not be applicable to client experiences

during other program years.

Table 3.1 shows that about 14 percent of the respondents knew about the program for less than one year,

19 percent for one year, 17 percent for two years, and 43 percent for three or more years. This shows that,

even during the ARRA period, many clients were familiar with the WAP program a long time before they

received WAP services.

Table 3.1 Awareness of Local Weatherization Program

How long have you known about your local weatherization program?

Number of Respondents

665

Less than 6 months

6-11 months

1 year

19%

2 years

17%

3-5 years

20%

6 or more years

23%

Don’t Know

TOTAL

100%

Respondents were asked an open-ended question about how they first learned about the local

weatherization program. Table 3.2 shows that clients were most likely to learn about the program from

family or friends (37 percent), and next most likely from the local agency (22 percent). Other common

methods were social services, the local media, a letter or flyer, or a neighbor who received services.

Table 3.2 How Learned about Local Weatherization Program

How did you find out about your local weatherization program?

Number of Respondents

641

Relative or friend mentioned the weatherization program

37%

Agency providing utility assistance, such as LIHEAP

22%

Social services/senior program

13%

Local newspaper or other local media

Letter/flyer

Neighbor who had their home weatherized

Utility company/fuel supplier

A call from the weatherization agency

Found the program on the internet

Church

Other

Table 3.3 shows that most households had to wait to have their homes weatherized. Even during the

ARRA period, about 40 percent of households waited at least one year to have their home audited, and

about 15 percent waited at least two years.

Respondents could choose more than one response.

Table 3.3 Time Since First Request for Weatherization

How long ago did you first request that your home be weatherized?

Number of Respondents

661

< 6 months

28%

6-11 months

22%

1 year

23%

2 years

3 or more years

10%

Don’t Know

11%

TOTAL

100%

Considering that many households waited one year or more to receive services, it is surprising that client

satisfaction with the waiting period was so high (Table 3.4). Only eight percent of clients were

dissatisfied with the time between the request and the receipt of services, while 44 percent reported that

they were very satisfied.

Table 3.4 Satisfaction with Time Between Request and Receipt of Services

How satisfied are you with the length of time between your request to have your home weatherized and when it

actually was weatherized?

Number of Respondents

663

Very satisfied

44%

Satisfied

40%

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

TOTAL

100%

Clients reported that it was easy to request weatherization services. These findings are presented in Table

3.5. Forty percent said it was very easy to request that their home be weatherized and another 47 percent

said it was easy. Since many households are referred directly from the LIHEAP program to the WAP

program, this result is not unexpected.

Table 3.5 Ease of Weatherization Request

How easy was it to request that your house be weatherized?

Number of Respondents

657

Very easy

40%

Easy

47%

Neither easy nor difficult

Difficult

Very difficult

TOTAL

100%

Table 3.6 presents information on the reasons clients provided for applying to the weatherization

program. Respondents could select more than one response to this question and could volunteer

additional reasons. The two most commonly mentioned reasons were to reduce energy bills (78 percent)

and to improve the comfort of their home (67 percent). Far less common reasons, mentioned by about

half as many clients, were environmental reasons and the opportunity to receive free services.

Table 3.6 Reasons for Weatherization Application

Why did you apply for the Weatherization Assistance Program?

Number of Respondents

660

Reduce energy bills

78%

Make home more comfortable

67%

Support environmental efforts to conserve energy

37%

Receive free services

34%

Improve health and/or safety (volunteered)

Other

Table 3.7 shows that prior to receiving weatherization services more than 80 percent of clients took one or

more actions to weatherize their home. The most common actions were low-cost items such as putting

draft stoppers on windows or doors, or using some material to stop drafts. However, some clients took

more extensive actions; 10 percent replaced or repaired roofs, furnaces, and doors or windows. Another 9

percent installed insulation. About one in five households reported that they had not done anything.

Respondents could provide more than one response for this question.

Table 3.7 Actions Taken by the Household Prior to Weatherization

Prior to receiving weatherization services, in what ways did your household attempt to weatherize your home?

Number of Respondents

665

Installed draft stoppers on doors or windows

34%

Used foam tape (or other material) to stop draft

33%

Engaged in energy efficient behaviors

11%

Replace or repaired roof/door/windows/appliances

10%

Added insulation

Replaced or repaired broken windows

Caulked holes and/or leaks

Put in insulation around a/c or heating ducts

Reduced heat/used space heaters

Did not do anything

19%

Other

Don’t Know

Overall, clients seemed to have a good understanding of the weatherization program and realistic

expectations of the weatherization outcomes. Even though they had to wait a long time for program

services, they reported being satisfied with the application and intake procedures.

3.2 AUDIT

When an agency is ready to weatherize a home, they start the process by conducting an audit of the

client’s home. In an audit, the agency sends an energy expert to interview the client, inspect the home,

and conduct tests to assess what energy efficiency measures would be cost-effective. In addition, many

agencies use this time in the home to educate clients about ways that they can save energy through

behavioral changes.

This section presents respondents’ views on the following aspects of the audit.

 Ease of scheduling

 Timeliness of auditors

 Courteousness of auditors

Almost all clients reported satisfaction with these aspects of the audit. Over 90 percent of clients reported

that it was very easy or easy to schedule the initial audit (Table 3.8), 95 percent reported that the auditor

was either early or on time (Table 3.9), and 98 percent them reported that the auditor was courteous or

very courteous (Table 3.10). These are high levels of satisfaction for service delivery professionals.

Respondents could provide more than one response for this question.

Table 3.8 Ease of Initial Audit Scheduling

How easy was it to schedule the initial audit of your home?

Number of Respondents

660

Very easy

49%

Easy

45%

Neither easy nor difficult

Difficult

Very difficult

TOTAL

100%

Table 3.9 Timeliness of Auditors

How timely were the people who did the initial audit of your home?

Number of Respondents

651

Early or on time

95%

Less than 30 minutes late

30-59 minutes late

1 to 3 hours late

More than 3 hours late

<1%

Did not show up on scheduled day

<1%

TOTAL

100%

Table 3.10 Courteousness of Auditors

How courteous were the people who did the initial audit of your home?

Number of Respondents

660

Very courteous

81%

Courteous

17%

Neither courteous nor rude

Rude

<1%

Very rude

TOTAL

100%

3.3 WEATHERIZATION SERVICES

If the auditor determines that there are weatherization measures that have a savings to investment ratio

(SIR) of 1.0 or greater, and that there are no barriers to installing the needed measures (e.g., structural

problems), the agency works with the client to schedule a time to deliver weatherization services. The

service delivery phase may take as little as one day on site, or as much as one week. In addition, certain

measures may be installed by other contractors and may need to be scheduled at a different time. For

example, not all weatherization agencies are qualified to install a new furnace and that measure may be

scheduled by a heating contractor. Each client is likely to have a somewhat different experience with the

delivery of weatherization services. Therefore, any observed variation in client satisfaction may relate to

differential performance, but also may relate to the types of measures installed.

This section presents respondents’ views on aspects of weatherization service delivery.

 Ease of scheduling services

 Timeliness of crew

 Courteousness, care, and cleanliness of the crew

 Satisfaction with condition of the home after services

Table 3.11 shows that 53 percent of clients said it was very easy to schedule a time for the weatherization

crew to come to their home. An additional 40 percent said it was easy. Less than three percent found this

process difficult or very difficult.

Table 3.11 Ease of Weatherization Installation Scheduling

How easy was it to schedule the time for the weatherization crew to come to your home?

Number of Respondents

661

Very easy

53%

Easy

40%

Neither easy nor difficult

Difficult

Very difficult

<1%

TOTAL

100%

Weatherization crew were rated very well on timeliness, as shown in Table 3.12. Ninety-three percent

said the crew was either on time or early and three percent said they were less than 30 minutes late. Less

than five percent reported that the crew was 30 minutes late or more.

Table 3.12 Timeliness of Weatherization Crew

How timely was the weatherization crew?

Number of Respondents

658

Early or on time

93%

< 30 minutes late

30-59 minutes late

1-3 hours late

> 3 hours late

Did not show up on scheduled day

<1%

TOTAL

100%

Table 3.13 shows that the weatherization crew was perceived as very courteous by the majority of clients.

Seventy-three percent said they were very courteous and 23 percent said they were courteous. Less than

one percent found them rude.

Table 3.13 Courteousness of Weatherization Crew

How courteous was the weatherization crew?

Number of Respondents

660

Very courteous

73%

Courteous

23%

Neither courteous nor rude

Rude

<1%

Very rude

TOTAL

100%

Table 3.14 shows that 63 percent of clients found the weatherization crew to be very careful and 30

percent said they were careful with the respondents' belongings. About four percent said they were

careless or very careless.

Table 3.14 Carefulness of Weatherization Crew

How careful of your home and belongings was the weatherization crew?

Number of Respondents

659

Very careful

63%

Careful

30%

Neither careful nor careless

Careless

Very careless

TOTAL

100%

Findings on how clean the crew left the inside and outside of the home are presented in Table 3.15. A

slightly larger percentage of clients rated the cleanliness of the outside of the house as very clean

compared to the inside of the house. Forty-seven percent of clients said the crew left the inside of the

house very clean and 42 percent said clean. In comparison, 55 percent of clients said the crew left the

outside of the home very clean and 37 percent said it was left clean. In both cases, three percent or less

said the crew left the home dirty.

Table 3.15 Cleanliness of Crew

Overall, how clean did the weatherization crew leave the inside/outside of your home?

Number of Respondents

Inside (664)

Outside (659)

Very clean

47%

55%

Clean

42%

37%

Neither clean nor dirty

Dirty

Very dirty

<1%

TOTAL

100%

Table 3.16 displays the results for client satisfaction with the final condition of the inside and outside of

the home after weatherization. Fifty-eight percent of clients were very satisfied with the condition of the

inside of the home, and 35 percent were satisfied. Similarly, 59 percent of clients were very satisfied with

the condition of the outside of the home after the weatherization work was done, and 34 percent were

satisfied.

Table 3.16 Satisfaction with Final Home Condition

Overall, how satisfied were you with the final condition that the inside/outside of your home was left in?

Number of Respondents

Inside (665)

Outside (659)

Very satisfied

58%

59%

Satisfied

35%

34%

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

<1%

TOTAL

100%

3.4 FINAL INSPECTION

After the weatherization work is completed, an inspector from the agency visits the home and checks to

make sure that all of the measures were correctly installed and are operating properly. During this visit,

inspectors usually have a discussion with the client about what was done in the home, what maintenance

of the installed measures is required (if any), and whether the client has any outstanding concerns. For

many agencies, the inspector also is responsible for delivering energy education.

The specific survey questions related to the final inspection included:

 Whether or not the final inspection had occurred

 Ease of scheduling the final inspection

 Timeliness of the crew

 Courteousness of the crew

Table 3.17 shows that 76 percent of clients had the final inspection of the home after the weatherization

work was done, 19 percent had not had the final inspection at the time of the survey, and 5 percent did not

know if it had taken place or not.

Table 3.17 Final Inspection Received

Have you had a final inspection?

Number of Respondents

665

Yes

76%

19%

Don’t Know

TOTAL

100%

As seen in Table 3.18, among those clients who had the final inspection, 62 percent found it very easy to

schedule, and an additional 35 percent found it easy to schedule.

Table 3.18 Ease of Final Inspection Scheduling

How easy was it to schedule the final inspection of your home?

Number of respondents who had a final inspection

502

Very easy

62%

Easy

35%

Neither easy nor difficult

Difficult

Very difficult

TOTAL

100%

Table 3.19 shows that 97 percent of clients who had a final inspection said that the people who did the

inspection were either early or on time.

Table 3.19 Timeliness of Final Inspector

How timely were the people who did the final inspections of your home?

Number of respondents who had a final inspection

501

Early or on time

97%

< 30 minutes late

30 - 59 minutes late

<1%

1 - 3 hours late

<1%

> 3 hours late

Did not show up on scheduled day

TOTAL

100%

Table 3.20 shows that of those who had a final inspection, the inspectors were perceived as very

courteous by 75 percent, and courteous by an additional 22 percent.

Table 3.20 Courteousness of Final Inspector

How courteous were the people who did the final inspection of your home?

Number of respondents who had a final inspection

504

Very courteous

75%

Courteous

22%

Neither courteous nor rude

Rude

<1%

Very rude

TOTAL

100%

4. CLIENT EDUCATION

This section of the report furnishes findings on the extent to which clients received education services that

helped them to understand ways to save energy and to improve health and safety in the home. The goal of

the WAP program is to make client homes safer and more affordable. To achieve that, the program is to

have trained professionals assess the performance of the home and to identify and install cost-effective

energy efficiency measures and necessary health and safety measures. However, it also is important for

clients to be active partners in saving energy and improving health and safety. The Satisfaction Survey

asked clients to describe the extent to which the program furnished information on energy efficiency and

health and safety, and to report on their satisfaction with that component of the program.

 Energy Education - The survey found that most, but not all, clients received energy education

services. When they received services, the clients generally reported that the information was

useful. However, compared to the investment in installation of energy efficiency measures and

resolution of health and safety problems, the investment in direct energy education was modest

and did not have a major impact on the client’s weatherization experience. Most clients reported

that they had less than 30 minutes of energy education. It is important to note, however, that the

survey did not attempt to capture the on-going education that takes place during interactions

between the client and different weatherization staff that come into the client’s home.

 Health and Safety Education - The survey found that less than half of the weatherization clients

reported that they had significant interactions with the weatherization program staff in terms of

health and safety issues. Among those clients who had in-depth discussions about health and

safety issues, it is clear that the clients understood some important messages about how to protect

their health and the health of their families, both in terms of the safety of energy efficiency

equipment and in terms of the impact of other factors on the indoor air quality in their homes.

However, the service delivery process did not focus as much on health and safety as it did on

energy efficiency.

These findings might suggest that the program could do more client education, particularly for health and

safety issues. However, the survey only tells us about the experience of clients, not the reason for those

experiences. With respect to energy efficiency education, many service delivery professionals suggest that

they can be more useful to clients by engaging them in the weatherization process, rather than by talking

about energy savings practices independent of the installed weatherization measures. In terms of health

and safety education, service delivery staff may only focus on these issues for homes and households

where there clearly is a problem. In both the Baseline Occupant Survey and in the Satisfaction Survey, the

majority of clients did not report any major health issues or safety concerns. One possible interpretation

of the results is that service delivery staff are responding to education opportunities, rather than

conducting the same education with every client.

Overall, many clients are getting energy education and some clients are receiving health and safety

education. While most clients seem satisfied with those services, it is not clear that they see this as a

major benefit of the program. And, even for those clients who received education, it does not seem as if

there has been a significant investment in terms of the time spent on it. The findings from this survey

suggest that client education in the program is treated as client service that supports the weatherization

process, rather than being a major focus of program delivery staff. If policymakers perceive that either

energy education or health and safety education can deliver major benefits to low-income households, it

seems that some change in either the delivery of those services or in the program guidance would be

required to achieve level of benefit.

4.1 ENERGY EDUCATION

This section reviews client perceptions about the amount, content, and usefulness of energy education

information provided by the program. The survey asked clients about discussions they had with the

service delivery staff and the materials that they received from program staff. It also asked clients to

report how much time they spent reviewing materials they received.

Table 4.1 shows that 69 percent of clients reported that they received information on ways to save energy

in the home during the time of the weatherization visit. This included both discussions with service

delivery staff and materials furnished by staff to the clients.

Table 4.1 Information Provided on Ways to Save Energy

Did the weatherization staff provide you with any information on ways to save energy in your home at the time of the

visit?

Number of Respondents

665

Yes

69%

23%

Don’t Know

TOTAL

100%

Table 4.2 presents results on the amount of time weatherization staff spent talking to clients about ways to

save energy. About one-fourth of clients reported that staff spent 30 minutes or more talking to them

about ways to save energy. But, the survey also found that about one-fourth of clients reported that staff

did not spend any time discussing ways to save energy.

Table 4.2 Time Spent Talking About Ways to Save Energy

How much time did the weatherization staff spend talking to you about ways to save energy?

Number of Respondents

645

No Time

< 5 minutes

5-14 minutes

17%

15-29 minutes

25%

30- 60 minutes

18%

> 1 hour

No information provided

24%

Don’t know if information provided

TOTAL

100%

Among those customers who spoke to staff about ways to save energy, most reported that they understood

the information provided. Table 4.3 shows that among those who received information about energy

saving, 70 percent said they understood the information very well and an additional 26 percent said they

understood the information well.

Table 4.3 Understanding of Verbal Information Provided

How well did you understand what the weatherization staff said to you about saving energy?

Number of respondents who received information from staff on ways to save energy

457

Very well

70%

Well

26%

Neither well nor not well

Not well

<1%

Not well at all

No time spent talking about saving energy

TOTAL

100%

As seen in Table 4.4, 48 percent of clients reported that they received printed materials from the

weatherization staff on saving energy. These printed materials were in the form of brochures, booklets or

manuals. Two percent reported receiving other materials such as CDs, DVDs, or hardware kits of

weatherization materials.

Table 4.4 Staff Provided Educational Materials about Saving Energy

What educational materials, if any, did the weatherization staff give you about saving energy?

Number of respondents

665

One or more brochures, booklets, or manuals

48%

Other

No materials were provided

36%

Don’t know if materials were provided

15%

Table 4.5 provides information on the amount of time spent reviewing materials. Over one-third of

clients reported that they received materials and spent at least 15 minutes reviewing them. However, over

one-half of the clients reported that they did not receive materials or they do not remember if they

received any materials.

Table 4.5 Respondent Time Spent Reading or Reviewing Materials about Saving Energy

How much time have you spent reading or reviewing the materials about saving energy that the weatherization staff

gave you?

Number of respondents

665

No time

< 5 minutes

5-14 minutes

10%

15-29 minutes

16%

30-60 minutes

15%

> 1 hour

No materials provided

36%

Don’t know if received materials

15%

TOTAL

100%

Table 4.6 shows that respondents had a good understanding of the materials weatherization staff provided

on ways to save energy. Sixty-six percent of clients who received printed materials said they understood

the materials very well, and 30 percent reported they understood them well.

Respondents could provide more than one response for this question.

This includes respondents who reported ‘don’t know’ when asked whether weatherization staff provided any information on

saving energy.

This includes those who reported that the staff member only spent time demonstrating how to save energy.

This includes respondents who reported ‘don’t know’ when asked whether weatherization staff provided any information on

saving energy and those who replied ‘don’t know’ when asked specifically whether weatherization staff provided any educational

materials about saving energy.

Table 4.6 Respondent Understanding of Materials about Saving Energy

Staff-provided materials on energy savings were also perceived to be useful by clients, as shown in Table

4.7. Thirty-nine percent of clients who received printed materials found the materials very useful and 52

percent found them useful.

Table 4.7 Usefulness of Materials about Saving Energy

How useful have the energy savings materials been to you?

Number of respondents who received printed materials from staff on ways to save energy

317

Very useful

39%

Useful

52%

Neither useful nor not useful

Not useful

Not useful at all

No time spent reading materials

TOTAL

100%

Respondents were asked what they found particularly useful about these materials. Table 4.8 shows that

respondents were most likely to report that the tips for saving energy were most useful (61 percent).

Other features that were reported to be useful included actual energy saving and easy to follow

instructions.

How well did you understand the energy savings materials that the weatherization staff gave you?

Number of respondents who received printed materials from staff on ways to save energy

319

Very well

66%

Well

30%

Neither well nor not well

Not well

Not well at all

No time spent reading materials

TOTAL

100%

Table 4.8 Useful Features of Materials about Saving Energy

What about the materials were particularly useful?

Number of respondents who received printed materials and found it useful

286

Useful tips

61%

Actual energy savings

32%

Easy to follow instructions

18%

Non-energy tips

Other

Don’t Know

11%

Respondents who found the materials on energy savings to be less than very useful were asked what

could be done to improve these materials. Table 4.9 shows the suggestions respondents provided for how

materials on energy savings could be improved. Sixty-seven percent could not provide any suggestions

for improvement, and 24 percent did not know how to improve the materials. The suggestions that were

provided included more information, pictures, and more discussion with staff.

Table 4.9 Ways to Improve Materials about Saving Energy

How could the materials have been improved for your use?

Number of respondents who received printed materials and found it less than very useful

187

Contained more information

Been discussed more by weatherization staff

Provided pictures

No suggested improvements

67%

Other

Don’t Know

24%

Results on clients' overall satisfaction with information on ways to save energy are presented in Table

4.10. Almost all clients who receive information on ways to save energy were either satisfied or very

satisfied with the information.

Respondents could provide more than one response for this question.

Table 4.10 Overall Satisfaction with Energy Saving Information Provided

How satisfied are you with the ways that the weatherization staff provided you with information about saving energy?

Number of respondents who received information from staff on ways to save energy

457

Very satisfied

54%

Satisfied

40%

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

TOTAL

100%

With respect to energy education, the survey found that most, but not all, clients received energy

education services. When they received services, the clients generally reported that the information was

useful. However, it seemed that, compared to the investment in installation of energy efficiency measures

and resolution of health and safety problems, the investment in direct energy education was modest and

did not have a major impact on the client’s weatherization experience. Most clients reported that they had

less than 30 minutes of energy education.

4.2 HEALTH AND SAFETY EDUCATION

This section reviews client perceptions about the amount, content, and usefulness of health and safety

information provided by the program. The survey specifically asked clients whether weatherization staff

engaged them in discussions about health and safety issues, whether staff furnished materials, and

whether they were referred to other services.

Table 4.11 shows that 36 percent of clients reported that they were asked about the health of household

members by weatherization staff and 14 percent said that they provided household member health status

information to the staff. However, in 37 percent of cases, no information about the health of household

members was sought by weatherization staff or provided to them.

Table 4.11 Staff Knowledge about Health of Household Members

Did the weatherization staff ask you/did you provide them with any information about the health of the members of

your household at any time during the process?

Number of Respondents

665

Staff asked about health of household members

36%

Respondent provided information about health of household members

14%

Staff didn’t ask/respondent didn’t provide information about health of household members

37%

Don’t Know

14%

TOTAL

100%

Table 4.12 shows that among those clients who were asked or provided information about household

member health to program staff, only five percent were in need of any health care at the time of the

weatherization visit. Four of those 16 clients received help from weatherization staff to get the needed

health care.

The responses to Table 4.12 furnish some context for the findings in Table 4.11. Among those households

who were specifically asked about health issues, only 5 percent reported that they had unmet needs. If

clients appear to have such a low rate of unmet needs, it is easy to understand why weatherization staff

might feel that it was important to focus on other issues in their discussions with clients. However, even

though this is a small percentage, it is appropriate for weatherization staff to ask screening questions

related to health of household members so that those issues are not overlooked.

Table 4.12 Assistance for Household Members in Need of Health Care

When the work crew came to your home, were any members of your household in need of health care, but not

receiving it?

Number of respondents who were asked by staff or provided staff information about health of

household members

331

Household members in need of healthcare

No household members in need of healthcare

95%

TOTAL

100%

Table 4.13 shows that referrals for social services were more common than help with health care needs.

Eleven percent of clients reported that weatherization staff referred them to social service programs. The

most common referral was LIHEAP. Other referrals were made to housing assistance, food assistance

programs, other energy programs, or roof repair, but these were less common.

Note: In 2 cases the respondent should have received the question on the provision of information to the household, but did not.

In these 2 cases, the respondent did give an answer to whether household members were in need of healthcare; these 2 cases are

included in the base of both of these tables.

Table 4.13 Referrals to Social Service Programs

Did the weatherization staff or your weatherization agency refer you to any other housing and/or social service

programs?

Number of Respondents

650

Staff referred household to social services

11%

Staff did not refer client to any housing and/or social services

89%

TOTAL

100%

What program or programs did they refer you to?

Number of Respondents

640

Low-Income Energy Assistance Program (ex. LIHEAP)

Housing assistance program

Food stamps and other food assistance programs

Roof Repair

Another Energy Program

Medical care (MEDICAID, CHIP, etc)

Did not refer client to any housing and/or social services

89%

Other

Table 4.14 shows that 40 percent of clients reported weatherization staff provided them with information

on ways to improve health and safety in their homes. Table 4.15 shows that, among those clients who

received some information, most reported that the amount of time spent was less than 15 minutes. Only

one in five clients reported that they discussed health and safety issues for 15 minutes or more.

Table 4.14 Staff Provided Information on Ways to Improve Health and Safety in the Home

Did the weatherization staff provide you with any information on ways to improve health and safety in your home?

Number of Respondents

665

Yes

40%

43%

Don’t Know

17%

TOTAL

100%

Respondents could provide more than one response for this question.

Table 4.15 Staff Time Spent Talking About Improving Health and Safety

How much time did the weatherization staff spend talking to you about ways to improve health and safety?

Number of Respondents

662

No time

< 5 minutes

5-14 minutes

14%

15-29 minutes

14%

30-60 minutes

> 1 hour

No information provided

44%

Don’t know if received health and safety information

17%

TOTAL

100%

Table 4.16 shows that clients who reported that staff provided verbal information about ways to improve

health and safety in the home said they understood what they were told. Seventy-one percent understood

the information very well and 27 percent understood it well.

Table 4.16 Respondent Understanding of Information on Improving Health and Safety

How well did you understand what the weatherization staff said to you about improving health and safety?

Number of respondents who received information from staff on ways to improve health and

safety

263

Very well

71%

Well

27%

Neither well nor not well

Not well

Not well at all

No time spent talking with weatherization staff

TOTAL

100%

About 40 percent of clients reported that they received verbal information on health and safety from the

staff. However, Table 4.17 shows that fewer clients reported that they received printed materials on this

topic. Twenty-two percent reported they were given brochures, booklets, or manuals. An additional two

percent recalled that they received some other types of materials.

This refers to respondents who reported "don’t know" when asked whether weatherization staff provided any information on

health and safety.

Table 4.17 Health and Safety Materials Provided

What materials, if any, about improving health and safety did the weatherization staff give you?

Number of Respondents

664

One or more brochures, booklets and manuals

22%

Other

No materials were provided

53%

Don’t know if health and safety materials were provided

24%

Table 4.18 shows that ten percent of respondents said they spent less than 15 minutes reviewing materials

on improved health and safety, eight percent spent 15 to 29 minutes, and five percent spent 30 minutes or

more. Seventy-seven percent were not provided with (or did not recall receiving) any printed materials

on this topic.

Table 4.18 Time Reviewed Health and Safety Materials

How much time have you spent reading or reviewing the materials about improving health and safety that the

weatherization staff gave you?

Number of Respondents

664

No time

< 5 minutes

5-14 minutes

15-29 minutes

30-60 minutes

> 1 hour

No materials provided

53%

Don’t know if materials provided

24%

TOTAL

100%

Respondents could provide more than one response to this question.

This includes respondents who reported "don’t know" when asked whether weatherization staff provided any information on

health and safety.

This includes respondents who reported "don’t know" when asked whether weatherization staff provided any information on

health and safety, and those who reported don’t know when asked specifically whether weatherization staff provided any printed

materials on health and safety.

Table 4.19 shows clients’ reports on understanding of the provided materials. The table shows that 68

percent said they understood the materials very well, and 28 percent understood them well.

Table 4.19 Respondent Understanding of Materials Provided About Improving Health and Safety

How well did you understand the materials about improving health and safety that the weatherization staff gave you?

Number of respondents who received printed materials from staff on ways to improve health

and safety

155

Very well

68%

Well

28%

Neither well nor not well

Not well

Not well at all

No time spent reading the materials

TOTAL

100%

Respondents who received printed materials on health and safety stated that they found them useful.

Table 4.20 shows that 44 percent found these materials on health and safety to be very useful, and 41

percent said they were useful.

Table 4.20 Usefulness of Materials about Improving Health and Safety

How useful have the materials about improving health and safety been to you?

Number of respondents who received printed materials from staff on ways to improve health

and safety

155

Very useful

44%

Useful

41%

Neither useful nor not useful

Not useful

Not useful at all

No time spent reading the materials

TOTAL

100%

Respondents who indicated that the printed materials on improving health and safety were useful were

asked to list the features they found particularly useful about them. These results are presented in Table

4.21. The two most commonly mentioned features were the warnings about dangerous materials in the

home, mentioned by 32 percent of respondents and warnings about fire and CO hazards, mentioned by 30

percent of respondents. Fifteen percent of indicated that information on the proper use of new heating or

cooling equipment was particularly useful. Twelve percent mentioned energy saving tips. Other features

were mentioned by less than ten percent of respondents.

Table 4.21 Useful Features of the Materials on Improving Health and Safety

What about the materials was particularly useful?

Number of respondents who found printed materials from staff on ways to improve health

and safety useful

131

Warned about dangerous materials in home

32%

Warned about fire/CO hazards

30%

Gave proper use on new heating/cooling equipment

15%

Provided energy saving tips

12%

Gave occupant specific advice

Warned about building structure and roofing

Gave information on potential pest problems

Other

14%

Don’t Know

14%

Respondents were also asked for their suggestions on ways to improve these materials. Table 4.22 shows

that more than half had no suggestions for improvements. Among the suggestions were that the materials

should include more information and that the materials should be discussed more by the weatherization

staff.

Table 4.22 How Materials on Health and Safety could be Improved

How could the materials have been improved for your use?

Number of respondents who read printed materials from staff on ways to improve health and

safety and found them less than very useful

Contained more information

Been discussed more by weatherization staff

Been bilingual

No suggested improvements

58%

Other

Don’t Know

33%

Results on overall client satisfaction with printed materials on ways to improve health and safety in the

home are presented in Table 4.23. More than half, 55 percent, said they were very satisfied with these

materials, and an additional 40 percent were satisfied.

Respondents could provide more than one response for this question.

Table 4.23 Satisfaction with Health and Safety Information

How satisfied are you with the ways that the weatherization staff provided you with information about improving

health and safety?

Number of respondents who received information from staff on ways to improve health and

safety

263

Very satisfied

55%

Satisfied

40%

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

TOTAL

100%

Table 4.24 displays information on respondent suggestions for ways to improve how the program

provides health and safety information. Eleven percent suggested that the agency should provide more

information. Twenty-one percent could not provide an answer to this question and 9 percent had no

suggestion. Fifty-five percent were very satisfied with the way the weatherization staff provided health

and safety information.

Table 4.24 Potential Improvements to Health and Safety Information

How could the agency improve the ways that it provides health and safety?

Number of respondents who received information from staff on ways to improve health and

safety

264

More literature/information

11%

Other

No improvement

Don’t Know

21%

Very satisfied with how information provided

55%

With respect to health and safety education, the survey found that only some clients received health and

safety information. When they received such information, the clients generally reported that it was useful.

However, it seems that, compared to the investment in installation of energy efficiency measures and

resolution of health and safety problems, the investment in direct education on health and safety issues

was limited and did not have a major impact on the clients' weatherization experiences.

Respondents could provide more than one response for this question.

5. OVERALL PROGRAM SATISFACTION

The previous sections of this report present information on the individual components of the program,

including each step in the weatherization process and the different types of education services delivered to

clients. This section presents findings on the client satisfaction with and assessment of the services that

the program delivered. The specific topics include:

 Areas for Improvement – A small percentage of the clients reported problems with installed

measures and some perceived that their home could use additional measures.

 Overall Program Satisfaction - Clients report high levels of satisfaction with the quality of the

work, with the performance of the installed equipment, and with the program overall.

 Satisfaction Benchmarks – Looking at both overall satisfaction and satisfaction with the

individual program components, it is possible to see where the program is performing at the highest level,

and where improvement could be made.

 Benefits – Clients reported that the program improved their comfort, lowered their energy bills,

and made them feel more confident that they could afford to stay in their home.

The overall picture presented by the Satisfaction Survey is that the WAP program is very successful in

meeting client needs. The program could do more to educate clients on energy efficiency opportunities

and on way to improve health and safety. In addition, it might be possible for the program to give clients a

better understanding of why certain measures are selected and others are not. But overall, clients perceive

that the program is meeting their needs.

5.1 AREAS FOR IMPROVEMENT

Respondents were asked if they had any problems with the weatherization services. Table 5.1 shows that

only ten percent of clients reported any problems with the services.

Table 5.1 Problems with Weatherization Services Received

Did you have any problems with the weatherization services you received?

Number of Respondents

664

Yes

10%

90%

TOTAL

100%

Respondents who had a problem with weatherization services were asked several follow-up questions

about the problems they encountered and how they dealt with them. Table 5.2 shows that 30 percent

(twenty clients) filed a complaint about the services.

Table 5.2 Complaint Filed about Weatherization Services

Did you file a complaint about the weatherization services provided?

Number of respondents who had a problem with weatherization services received

Yes

30%

70%

TOTAL

100%

Respondents who filed a complaint were asked additional questions about the nature of the complaint,

their satisfaction with how the complaint was handled, and what actions the agency could have taken to

resolve the complaint. When asked about the nature of the weatherization complaint, half of the 20

respondents stated that one reason for the complaint was the measure installation, and nine respondents

mentioned the materials and/or equipment. Three of the 20 respondents mentioned the crew as a reason

for the complaint. Three respondents mentioned clean-up, and three had complaints about other aspects of

the program.

Seven of the 20 respondents who filed a complaint reached a resolution of the situation about which they

complained. One respondent was very satisfied with the resolution, three were satisfied, and three were

neither satisfied nor dissatisfied. Of the 19 respondents who were not very satisfied with the resolution of

their complaints, 12 stated that the agency could still act on their complaint to fix the problem. Three

respondents felt the agency could have listened better. One respondent felt they could not have done

anything differently, and two respondents did not know how the agency could have done a better job to

resolve the complaint. Four respondents had other things they thought the agency could have done as

well.

Respondents were asked if there were any other measures or equipment they thought should have been

installed to help them save energy. Table 5.3 shows that 33 percent reported there were other measures or

equipment they felt the weatherization staff should have installed in their home to save energy. Far more

clients perceived that they needed additional energy saving measures than had problems with the

measures installed. However, since measure installation decisions in the program are guided by cost-

effectiveness calculations, it is possible that clients would not understand why certain measures were not

installed.

Table 5.3 Household Perceived Need for Additional Measures

Do you feel that any additional measures or equipment should have been installed in your home to help you save

energy?

Number of Respondents

643

Yes

33%

67%

TOTAL

100%

The most commonly mentioned additional measure was window caulking, replacement, or repair, which

was mentioned by 14 percent (Table 5.4). Insulation and door replacement or repair was mentioned by 5

percent. Other measures or equipment were mentioned by fewer than 5 percent. Since, in most cases,

window replacement is not a cost-effective energy conservation measure (i.e., does not have a savings-to-

investment ratio of 1.0 or greater), but clients perceive that new windows would save energy, it is not

surprising that almost one-half of client concerns are about windows.

Table 5.4 Additional Weatherization Measures

What other measures or equipment should have been installed?

Number of Respondents

637

Window caulking, window repair or replacement

14%

Insulation

Door repair or replacement

Furnace repair or replacement

Refrigerator

Other appliances

Air conditioner repair or replacement

Water heater repair or replacement

Ventilation repair or replacement

Weather stripping

Roof repair or replacement

Air-sealing

Other

No additional measures needed

68%

Respondents could provide more than one response for this question.

Another challenge for the program is home repairs. Many homes need major repairs, but the program

often does not have the funding to make them. Table 5.5 shows that 46 percent of clients were aware that

the weatherization staff checked to see if major repairs were needed, 17 percent said their home needed

major repairs, and 12 percent reported the repairs were made. These statistics show that the program is

making repairs that allows delivery of weatherization services to proceed. However, in some homes, the

repairs were not completed. Presumably this was because the problem did not prevent the agency from

installing appropriate weatherization measures.

Table 5.5 Staff Checked for Major Repairs

Did the weatherization staff check your home for major repairs, such as roof repairs, at any time during the process?

Did they say repairs were needed in your home? Were major repairs done to your home?

Number of Respondents

665

Checked for repairs

46%

Repairs were needed

17%

Major repairs done to home

12%

Table 5.6 presents clients’ suggestions for program improvement. The table shows that 62 percent of

clients had no suggestions for program improvements. Nine percent suggested more funding and

approved installations, eight percent suggested better communication regarding the process, and eight

percent suggested improving the quality of materials, crew or inspections.

Table 5.6 Suggestions for Program Improvement

What suggestions, if any, do you have for how the weatherization program can be improved?

Number of Respondents

665

More funding/approved installations

Be more upfront with process/better communication

Improve quality of materials/crew/work/inspections

Faster weatherization process

No suggested improvements

62%

Other

Don’t Know

Respondents could provide more than one response for this question.

5.2 OVERALL PROGRAM SATISFACTION

In Section 3 of the report, clients reported high rates of satisfaction with the different stages of

weatherization service delivery. Clients report similar levels of satisfaction with the final outcome of the

service delivery process. Table 5.7 shows that clients were well satisfied with the weatherization work

that was done in their home. Sixty percent were very satisfied and 33 percent were satisfied.

Table 5.7 Overall Satisfaction with Work Done

Overall, how satisfied are you with the work performed in your home?

Number of Respondents

664

Very satisfied

60%

Satisfied

33%

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

<1%

TOTAL

100%

Similarly, Table 5.8 shows that clients were well satisfied with the new equipment installed in their home

through this program. Sixty-two percent reported they were very satisfied and 30 percent reported they

were satisfied.

Table 5.8 Overall Satisfaction with New Equipment Installed

How satisfied are you with any new equipment installed in your home?

Number of Respondents

647

Very satisfied

62%

Satisfied

30%

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

TOTAL

100%

Table 5.9 displays the findings on client satisfaction with the energy savings. Thirty-eight percent of

clients reported that they did not know if they had saved any energy after weatherization, 35 percent were

very satisfied with their energy savings, and 22 percent were satisfied. Three percent reported that they

did not save any energy.

Table 5.9 Satisfaction with Energy Savings after Weatherization

After having your home weatherized, did you save any energy?

Number of Respondents

665

Saved energy after weatherization

59%

Did not save energy after weatherization

Don’t Know

38%

TOTAL

100%

How satisfied are you with the energy savings achieved after having your home weatherized?

Number of Respondents

662

Very satisfied

35%

Satisfied

22%

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

Did not save energy

Don’t know if saved energy

38%

TOTAL

100%

Table 5.10 shows that overall program satisfaction was high. Sixty-three percent of clients reported they

were very satisfied with the weatherization program and 31 percent were satisfied.

Table 5.10 Overall Satisfaction with the Weatherization Program

Rate your overall satisfaction with the weatherization program.

Number of Respondents

665

Very satisfied

63%

Satisfied

31%

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

<1%

TOTAL

100%

5.3 SATISFACTION BENCHMARKS

Table 5.11 summarizes client satisfaction with various components of the program. The percentage of the

total sample of clients who said they were very satisfied with each component of the program is

presented. Clients reported the highest levels of satisfaction, around 60 percent very satisfied, with the

program overall, the new equipment installed, the work performed in the home, and the condition of the

inside and outside of the home the after weatherization work crew was done.

Table 5.11 Overall Satisfaction with Key Components of the Program

How satisfied are you with the ________?

Number of Respondents

665

Very Satisfied

Satisfied

Length of time between request to have home weatherized and when it was

weatherized

44%

39%

Final condition of the inside of the home

58%

35%

Final condition of the outside of the home

59%

33%

Work performed in the home

60%

33%

New equipment installed in the home

61%

30%

Energy savings achieved after having the home weatherized

35%

22%

Information on saving energy

37%

28%

Information on improving health and safety

22%

16%

Overall Weatherization Program

63%

31%

Client satisfaction was substantially lower for the length of time between the request for weatherization

and receipt of services, the way in which staff provided information about ways to save energy in the

home, and actual energy savings achieved in the home. Only 35 percent of clients were very satisfied with

the energy savings they achieved after weatherization and 22 percent were very satisfied with the way the

staff provided information on improving health and safety in the home.

Table 5.12 is a summary table showing client perceptions of the ease of dealing with different aspects of

the program. The percentages of clients who rated each part of the program as "very easy" or "easy" are

presented. Scheduling the final inspection appears to be the easiest part of the program; 62 percent said it

was very easy and 35 percent said it was easy. Fifty-three percent said scheduling the weatherization

crew was very easy, 40 percent said it was easy. Forty-nine percent rated the scheduling of the audit as

very easy, and 45 percent said this function was easy. Forty percent said the initial request for services

was very easy, and 47 percent said it was easy.

Table 5.12 Ease of Dealing with Different Aspects of the Program

How easy was it to ________?

Respondents

Very Easy

Easy

Request that the home be weatherized

657

40%

47%

Schedule the initial audit

660

49%

45%

Schedule the weatherization crew to come to home

661

53%

40%

Schedule the final inspection

502

62%

35%

Table 5.13 summarizes client perceptions of the timeliness of various components of the program. More

than 90 percent of clients said that program staff were either early or on time for the initial audit, the

weatherization services, and the final inspection.

Table 5.13 Timeliness of Various Components of the Program

How timely was/were ________?

Respondents

Percent who said

‘early or on time’

The people who did the initial audit of the home

651

95%

The weatherization crew

658

93%

The people who did the final inspection of the home

501

97%

5.4 OVERALL PROGRAM BENEFITS

Table 5.14 displays findings on client perceptions of the benefits of program participation. The three

benefits mentioned most often by respondents were improved comfort (47 percent) improved home

equipment (46 percent), and lower energy bills (43 percent).

Table 5.14 Benefits of Program Participation

What are some of the greatest benefits your household received by participating in the weatherization program?

Number of Respondents

639

More comfortable home

47%

Improved home equipment

46%

Lower energy bill

43%

Home Insulated

13%

Health benefits for household members

Environmental benefits

Tips given by staff

Other

Respondents could provide more than one response to this question.

Table 5.15 shows that many clients tell others about WAP. Seventy-nine percent of clients told others

about the program and 19 percent said the person they told received services. Twenty-three percent said

the person they referred did not receive services and 38 percent did not know the outcome of their

referral.

Table 5.15 Word of Mouth Referral

In the last 12 months, have you told other people who might be interested in receiving weatherization services about

the program?

Number of Respondents

659

Told others about the program

79%

Did not tell others about the program

21%

TOTAL

100%

Have those people had their homes weatherized, or are they scheduled to have their home weatherized, as a result

of your suggestion?

Number of Respondents

659

They got weatherized/are scheduled to be weatherized

19%

They did not receive services

23%

Did not tell others about the program

21%

Don’t Know

38%

TOTAL

100%

Tables 5.16 and 5.17 furnish statistics on the likelihood that a household will move from their

weatherized home. Table 5.16 shows that 80 percent of clients reported that they were less likely to move

from their home and 12 percent reported no change in their likelihood of moving. Table 5.13 shows that

58 percent of clients reported that they were very unlikely to move in the next 12 months and 27 percent

reported that they were unlikely to move. Only five percent were either very likely or likely to move in

the next year.

Table 5.16 Impact of Services on Likelihood of Moving

Do you think weatherization will have any impact on how long you stay in your current home? Would you say you are

________?

Number of Respondents

665

More likely to move from your home

Less likely to move from your home

80%

Equally likely to move from your home

12%

Already moved

Don’t Know

TOTAL

100%

Table 5.17 Likeliness Household Will Move in Next 12 months

How likely is your household to move in the next 12 months for any reason or combination of reasons?

Number of Respondents

658

Very likely

Likely

Neither likely nor unlikely

Unlikely

27%

Very unlikely

58%

No chance

Already moved

TOTAL

100%

Overall, clients perceive that the program has benefited them by making their homes more comfortable

and making it easier to stay in their homes. They tell other low-income households about these program

benefits and find that many of the people they tell about the program also receive services.