Customer Satisfaction Survey of the NHGRI/ORDR Genetic and Rare Diseases Information Center (GARD)

National Center for Advancing Translational Sciences –

Office of Rare Diseases Research

and

National Human Genome Research Institute,

National Institutes of Health

Genetic and Rare Diseases Information Center

Customer Satisfaction Survey

Final Report of Survey Data

December 2014

Report Prepared by

Lisa S. Lanier, Ph.D., M.P.H.

Evaluation Consultant

Table of Contents

Executive Summary ..............................................................................................................................................3

Report of Survey Data ..........................................................................................................................................6

Background .............................................................................................................................................................6

Survey Development ...............................................................................................................................................7

Survey Pretest .........................................................................................................................................................7

Data Collection Methodology .................................................................................................................................8

Results of Data Analyses ........................................................................................................................................8

Survey Objective 1. Who is being served by GARD ........................................................................................8

Survey Objective 2. How do individuals learn about GARD ......................................................................... 14

Survey Objective 3, 4, & 5. Information seeking – why, type, and other sources ......................................... 15

Survey Objective 6. Customer satisfaction ..................................................................................................... 17

Customer Satisfaction Survey Findings - 2006 vs. 2014 ...................................................................................... 21

Limitations ............................................................................................................................................................ 22

Recommendations ................................................................................................................................................. 24

Conclusion ............................................................................................................................................................ 24

Content Notes........................................................................................................................................................ 26

Appendix A: English and Spanish Web-Based Customer Satisfaction Survey ............................................ 27

Appendix B: Report of Survey Pretest Results ................................................................................................ 43

Appendix C: Email Response with Survey Information ................................................................................. 48

Appendix D: Results of Analysis of Spanish Responses .................................................................................. 51

Appendix E: Individual Suggestions for Improving Information Received ................................................. 58

from GARD and Additional Comments about GARD

Appendix F: Individual Compliments about GARD ....................................................................................... 67

Executive Summary

Background

The Genetic and Rare Diseases Information Center (hereinafter referred to as GARD) has been in operation for

approximately 12 years. Over the years changes have been made, based in part on results from a customer

satisfaction survey conducted in 2006 and feedback from GARD customers, to how GARD presents

information and how individuals access information from GARD. Currently, individuals can access information

from GARD by visiting the GARD web pages (located on the National Center for Advancing Translational

Sciences/Office of Rare Diseases Research website at http://rarediseases.info.nih.gov/GARD/), submitting an

online contact form located at http://rarediseases.info.nih.gov/gard/contact-gard, sending an email to

[email protected], sending a letter to Genetic and Rare Diseases Information Center, P.O. Box 8126,

Gaithersburg, MD 20898-8126, or calling a toll-free number (888-205-2311) to speak with an information

specialist.

The purpose of conducting a customer satisfaction survey in 2014 was to systematically collect data that could

be used, in part, to support changes to the information service to ensure that GARD continues to operate in a

cost effective manner and provide an effective, high quality service to a broad audience. A web-based survey

containing 29 questions was developed in English and translated to Spanish. The survey was designed to (1)

identify who is being served by GARD, (2) identify how individuals learn about GARD, (3) identify why

individuals seek information from GARD, (4) identify the type of information being sought, (5) identify where

else customers might have sought information, and (6) assess customers’ satisfaction with the information they

receive from GARD.

Training took place for phone interviewers and GARD staff. Five individuals participated in a survey pretest

(four in English and one in Spanish). No changes were made to the survey or data collection system following

the pretest. Approval to collect data was obtained through the National Institutes of Health from the Office of

Management and Budget and data collection commenced on March 10, 2014. Individuals who contacted GARD

by visiting the GARD web pages, submitting the online contact form, sending an email or letter, or calling

GARD self-selected to complete the survey in English or Spanish. Individuals who contacted GARD by phone

were invited to complete the survey by phone or online. Data collection ended April 25, 2014.

Survey Results

During the data collection period, 930 surveys were completed of which 98.0% were completed in English and

2.0% in Spanish.

Who is being served by GARD?

The main characteristics of survey respondents were as follows:

•

Patient or family member/friend of a patient

Female

Age 51-60

Bachelor’s degree

White

Prefer to speak and read in English

Reside in a suburban area

If reside in the United States, reside in one of the southern states

If reside outside of the United States, reside in Europe

How do individuals learn about GARD?

The majority (~ 64%) of survey respondents found out about GARD by conducting a search using a search

engine such as Google or Yahoo. Most (67%) survey respondents were one-time visitors to GARD in part of

2014 (i.e., between January 1, 2014 and April 25, 2014 when data collection ended).

Why do individuals seek information from GARD?

The top four reasons identified by survey respondents for contacting GARD were to obtain: (1) broad

information about a specific disease, (2) treatment or medical care information about a specific disease, (3) a

diagnosis based on symptoms, and (4) to find a research study or clinical trial for a specific disease.

Where else do GARD customers go to seek information about genetic and/or rare diseases?

The majority (71%) of survey respondents obtained information from another source before contacting GARD.

Survey respondents sought information from a wide variety of websites and were overall satisfied with the

information they received from other sources. Reasons for contacting GARD after seeking information

elsewhere were: to get more information overall and more in-depth information about a specific topic (e.g.,

clinical trials, treatment, etc.).

Are GARD customers satisfied with the information they receive?

Of the 33% of respondents who visited a disease-specific information page (hereinafter referred to as a Q&A)

on the GARD web pages, the majority (78%) indicated that they did not contact the Information Center after

reviewing a Q&A. Overall, these 78% of respondents were somewhat to very much satisfied with the

information they received from the Q&A on the GARD pages of the NCATS/ORDR website.

Survey respondents who received information from GARD by email or letter were very much to completely

satisfied with the information they received (as indicated by ~27-35% of respondents answering very much or

completely to each respective question related to satisfaction).

Survey respondents who received information by speaking with a GARD information specialist on the

telephone expressed being highly satisfied (as indicated by ~33-58% of respondents answering completely to 8

of the 10 questions related to satisfaction) with the information they received from an information specialist.

Survey respondents are most likely to share the information they received from GARD with others, such as a

health care provider, family member, co-worker, etc., or use the information to educate themselves or for

educational reports or presentations.

Slightly less than half (48%) of survey respondents would likely contact GARD again.

Suggestions and comments

Four hundred and forty-one survey respondents provided very insightful suggestions and comments for

improving the information they received from GARD. Those suggestions were at times prefaced with

compliments about the information from the GARD website and/or the GARD information specialists.

Comparison of Survey Results: 2006 vs. 2014

Some of the questions asked in the 2006 survey were also asked in the 2014 survey. Demographics of the

convenience sample who completed the 2006 survey were similar to the demographics of the convenience

sample who completed the 2014 survey with the exception of the following: age of inquirers (31-40 in 2006 and

51-60 in 2014), education (post-graduate degree in 2006 and bachelor’s degree in 2014), and use of GARD

again (definitely in 2006 and likely in 2014). The table below shows the comparison of relevant items.

Descriptor

Year of User Satisfaction Survey

2006

2014

Primary Gender

Female

Age of Inquirers

31-40

51-60

Education

Post-graduate

Bachelor’s

Ethnicity

Non-Hispanic or Latino

Self-described Race

White

Preferred Language

English

Role (top 2 roles)

Patient, Family member/friend of patient

Residence

U.S. – Southern & Western states

U.S. – Southern states

Visits to GARD (in year of

survey)

One

Find GARD

Internet search engine (Google, Yahoo)

Inquiry Reason (top 2

reasons)

Specific disease/condition,

treatment/medical care

Specific disease/condition,

treatment/medical care

Research before contacting

GARD

Yes

Plan for information

Share with others

Use GARD again

Definitely

Likely

Limitations

The 2014 evaluation of GARD had the following limitations:

•

Use of a convenience sample

Insufficient pretest data

No interim data for analysis

These limitations contribute to the lack of ability to generalize with confidence to the larger GARD population

and unknown introduction of bias in the responses.

Recommendations

The recommendations stem from the feedback provided by survey respondents. These recommendations can be

a springboard for GARD funding agencies and staff to explore other methods of obtaining more detailed

information about the experience individuals have with GARD and to further discussions about ways to

improve the GARD information service.

The recommendations are listed below with details provided in the Recommendations section of this report.

•

Conduct ongoing data collection

Conduct outreach

Improve inquiry responses

Improve functionality of the NCATS/ORDR GARD web pages

Conduct a review of other existing genetic and rare diseases resources

Conclusion

Based on the survey results, it can be inferred that GARD provides a valuable and useful service to the genetic

and/or rare diseases community. It is recommended that prior to the development of the next GARD work plan

that representatives from the funding agencies and GARD staff review the survey results, in particular the

suggestions and comments, and the recommendations as outlined in this report to determine which suggestions,

recommendations, and/or activities are feasible and appropriate.

Customer Satisfaction Survey of the

Genetic and Rare Diseases Information Center

National Center for Advancing Translational Sciences

October 2014

Report of Survey Data

Background

The Genetic and Rare Diseases Information Center (hereinafter referred to as GARD) has been in operation for

approximately 12 years. It was created by two agencies of the National Institutes of Health (NIH), the

NCATS/Office of Rare Diseases Research (ORDR) and the National Human Genome Research Institute

(NHGRI). GARD provides patients, family members or friends of patients, health care providers, researchers,

advocacy groups, the general public and other audiences with information in English and Spanish about genetic

and/or rare diseases.

Since its inception, GARD has made changes in how it delivers information. In 2008, GARD expanded its

service by creating a web presence (i.e., web pages located on the NCATS/ORDR website) that enables visitors

to conduct their own searches for information about genetic and/or rare diseases. Since 2008 changes have been

made to the web pages based on usability testing and other feedback provided by visitors to the site.

There are currently three ways that a person can conduct a search for information about a particular genetic

and/or rare disease. He can enter the name of the disease/condition in a search box, browse diseases by first

letter or number, or find diseases by category, e.g., eye diseases, skin disorders, etc.

In addition to the search function, the GARD web pages consist of other helpful information such as how to find

a disease specialist, tips for the undiagnosed, and financial aid resources. There is an About GARD web page

that provides information about who GARD can help, where to start searching for information, where GARD

gets its information, how to get more information, and how to contact GARD staff. The About GARD web page

also has links to materials on broader topics of interest such as an article for health-care professionals’

newsletters, a video for researchers, a GARD brochure and flyer, and information on how to find a specialist,

financial assistance, etc.

If a visitor is unable to find the name of his condition using the search function or needs additional information,

he can contact an information specialist at GARD. To contact an information specialist, the visitor can complete

and submit an online contact form, send an email, send a letter by U.S. mail (Genetic and Rare Diseases

Information Center, P.O. Box 8126, Gaithersburg, MD 20898-8126) or fax (301-251-4911), or call by dialing a

toll-free national (888-205-2311), TTY (888-205-3223) or international (301-251-4925) phone number between

12:00 noon and 6:00 p.m. Eastern Time. Requests for information submitted using the online contact form, U.S.

mail or fax are answered within 20-30 business days.

GARD tracks its inquiries and passively collects some data (e.g., role such as patient, family member or

physician, frequency of use of GARD, referral, country of origin and U.S. state) from individuals who seek

information from GARD. The information that is collected helps provide a better understanding of the

demographics of GARD customers. Some of the data currently collected by GARD is identified in this report

for comparative purposes to the survey data.

In 2006, a customer satisfaction survey of GARD was conducted. Although the survey methodology and

questions were reflective of how GARD provided its services at that time, some questions, in particular the

demographic questions, were asked in the 2014 customer satisfaction survey. This report includes a section that

compares the results of the 2006 survey to the results of the 2014 survey, as applicable.

The purpose of conducting the 2014 customer satisfaction survey was to systematically obtain data from GARD

customers to assess their satisfaction with the information they received and with the ways in which they are

now able to obtain information from GARD. The data will be used to make changes, as appropriate and

feasible, to the information services to ensure that GARD continues to operate in the most cost-effective manner

while providing an effective, high quality service to a broad audience, and thereby serve as a resource to the

rare and genetic diseases audience that is unique in the United States.

The implementation of the 2014 customer satisfaction survey was made possible with funding from the 2%

Evaluation Set-Aside Program, Office of Program Evaluation and Performance, National Institutes of Health

(http://dpcpsi.nih.gov/opep/set_aside_program). The survey was conducted by ICF Incorporated, LLC (prime

contractor) and Lisa Lanier, Ph.D. (consultant).

Survey data analysis was performed by Dr. Lanier completely

independent of ICF Incorporated and the GARD staff.

Survey Development

A survey was designed to achieve the following objectives:

•

Identify who is being served by GARD.

Identify how individuals learn about GARD.

Identify the reasons why individuals are seeking information.

Identify the type of information being sought.

Determine where customers might have sought information before or in addition to GARD.

Assess customers’ satisfaction with GARD services in terms of effectiveness (e.g., quality, usefulness,

personal applicability, utility, timeliness, understandability, sensitivity, amount of information, and format

of delivery) of responses provided by information specialists and information available on the

NCATS/ORDR GARD web pages.

The survey, which was made available in English and Spanish, consisted of 29 questions. Twenty-seven of the

questions were closed-response (i.e., pre-coded) and two were open-ended. See Appendix A for a copy of the

survey instrument.

Survey Pretest

The purpose of the pretest was to:

• Identify any survey questions that respondents did not understand or in some way found difficult to answer.

• Identify any survey formatting or response flow difficulties that respondents may have experienced.

The pretest was conducted July-August 2013 with the goal of obtaining no more than nine complete surveys

(approximately half in English and the other half in Spanish).

The pretest methodology was as follows:

•

Phone interviewers and GARD Information Specialists were trained.

The pretest targeted individuals who contacted GARD and subsequently received an email response to their

question/request for information after submitting the online contact form or a regular email. In the response

was a link to the pretest survey.

Pretest survey respondents were instructed to use the comment field at the end of the survey to not only

provide comments about their experience with GARD but also provide comments about how to improve the

survey.

Due to a low response rate from individuals receiving a response by email, the pretest target audience was

expanded to include individuals who contacted GARD by phone.

Due to the low response rate and therefore the length of time it was taking to obtain nine completed surveys,

the pretest was stopped after a total of five completes (four in English and one in Spanish).

The survey pretest did not result in any changes to the survey instrument or the data collection methodology. A

copy of the Report of Survey Pretest Results is located in Appendix B.

Data Collection Methodology

Generic clearance for the collection of routine customer feedback was obtained in February 2014. Data

collection commenced on March 10, 2014 and ended on April 25, 2014. Convenience sampling was used to

collect the data. All individuals who contacted GARD (by online contact form, email, phone, letter/fax, or

website visit) were invited and self-selected to complete the survey.

•

For individuals visiting the GARD web pages, a pop-up invited the person to take the survey and asked if he

wished to take the survey. If the respondent selected “no”, the pop-up disappeared and did not reappear

again. If the respondent selected “yes”, a new window opened taking the respondent to the survey.

Individuals who contacted GARD by email or the online contact form received their response, which

included a footnote that they will receive a follow-up email inviting them to take the survey. A second email

was sent later that same business day that explained the purpose of the survey and contained the URL to the

survey. See Appendix C for a copy of the emails in English.

Individuals who contacted GARD by phone were invited to complete the survey by phone or online (in this

case, the URL was provided verbally to the individual, as well as instructions on how to access the survey

on the GARD web page). For individuals who said they wanted to take the survey by phone, the GARD

information specialist collected their names and contact information. This contact information was

transferred via a secure web portal to an ICF-trained interviewer who called the respondents and

administered the survey. If a respondent did not want to take the survey, no information was collected.

Results of Data Analyses

This section of the report provides the analyses of the data collected from March 10 through April 25, 2014.

Note: The percent reported is the valid percent (i.e., only the percentage of respondents who answered the

question.

Number of Completed Surveys

During the data collection period, 930 surveys were completed. The number of completed surveys was

sufficient to perform the analysis and adhered to OMB requirements. Of the 930 surveys completed, 98.0%

(n=911) were in English and 2.0% (n=19) in Spanish. A separate analysis was performed on the data from the

19 surveys completed in Spanish. The results of the analysis are located in Appendix D.

Notes of comparison:

(1) During the data collection period, GARD received 582 inquiries by Web contact form, 233 by phone, 87 by email,

and 5 by letter. During that time period, there were 360,807 website users (i.e., visitors to the GARD web pages of the

NCATS/ORDR website). Although these figures could be used to determine the denominator for the calculation of a

response rate, the calculation of a response rate is not key since the survey results are not generalized to the larger

population of GARD customers even though we assume that the information is insightful.

(2) The percent of surveys completed in each language is similar to the trend data collected by GARD. From 2002-2014

the percentage of inquiries by language was 97.0% in English and 3.0% in Spanish. The percentage of surveys

completed in each language is also similar to the percentage of inquiries received in each language during the data

collection period of March-April 2014, which was 97.0% English, 2.0% Spanish, and 1.0% another language.

Survey Objective 1. Identify who is being served by the GARD Information Center

Responses to survey questions 4 and 21-29 (see Appendix A) were designed to provide a profile of the survey

respondents. The profile of the 930 respondents was as follows:

Role

Survey respondents identified their role (relationship to the inquiry) during their most recent interaction with the

Information Center. Table 1 shows the percent of respondents and the role in which they self-identified.

Table 1

Role

Percent

Patient

41.0%

Family or friend of patient

24.7%

Other

9.6%

Student

5.7%

Other (than the categories listed) health care provider or allied health professional

4.3%

Nurse

4.0%

Physician

3.5%

Researcher/Scientist

3.0%

Teacher or Professor/Educator

2.4%

Journalist/Media professional

0.4%

Genetic Counselor

0.4%

Other roles identified include:

•

Attorney

Consultant

Disability Adjudicator

Other health care providers or allied health professionals include:

•

Nurse Practitioner (n=5)

Social Worker (n=5)

Speech Therapist (n=4)

Audiologist (n=3)

EMT (n=2)

Librarian (n=2)

Occupational Therapist (n=2)

Physical Therapist (n=2)

Psychologist (n=2)

•

Approximately 65% of the respondents identified themselves as either a patient (41%) or a family

member or friend of a patient (~24%).

Note of comparison: The survey data is similar to the GARD trend data (passively collected by GARD) in that the two

largest categories of GARD customers are individuals who identify themselves as a patient or a family member or friend

of a patient.

Residence

80.0% of respondents resided in the United States (U.S.).

Figure 1 shows the percent of respondents (n=506) from each of the four census bureau-designated regions

of the U.S.

Figure 1

South = 34.8%

West = 28.1%

Midwest = 20.5%

Northeast = 16.2%

U.S. Territory (Puerto Rico) = 0.4%

•

18.3% of respondents indicated that they resided outside the U.S.

Figure 2 shows the percent of respondents (n=154) from each region of the world.

Figure 2

Europe = 35.1%

North America (other than the United States) = 18.8% (16.9% from Canada and 1.9% from Mexico)

Australia = 16.2%

Asia = 12.3%

South America = 7.1%

Africa = 5.2%

Middle East = 3.9%

Central America = 0.6%

Caribbean = 0.6%

Note of comparison: GARD trend data from 2002-2013 shows 76% of inquiries were domestic and 13% were

international.

• 40.5% of respondents indicated they lived in a suburban area, 35.4% lived in an urban area, and 21.6%

in a rural area.

Language

•

There was a positive correlation between the language that respondents prefer to speak and the language

they prefer for written materials (i.e., to read).

Table 2 shows the language that the respondents prefer to speak and read.

Table 2

Language

Communication

Speak

Read

English

92.5%

93.8%

Spanish

2.3%

1.9%

Other

1.9%

1.4%

Eighty percent of the survey respondents resided in the United States with over 60% of those respondents

residing in the Southern (~35%) and Western (~28%) regions of the U.S. Of the approximately 18% of

respondents who resided outside the U.S., Europe (~35%) and Canada (~17%) were the regions of the world

where just over 50% of the non-U.S. respondents resided. Approximately 40% of the U.S. and non-U.S.

respondents combined resided in a suburban area.

•

The other languages identified by respondents were: American Sign, Dutch, Filipino, Finnish, French,

Hebrew, Hindi, Korean, Norwegian, PA Dutch, Portuguese, Russian, and Swahili. The language

identified with most frequency was Portuguese (n=8).

•

Over 90% of survey respondents preferred to speak and read in English and approximately 2%

preferred to speak and read in Spanish. Just under 2% of respondents indicated that there were

other languages that they preferred. Of the 13 other languages identified, Portuguese was the

language identified with most frequency (n=8).

Gender, Age, and Education

76.0% of respondents were female and 19.6% were male.

The tables below show the percent of respondents by age and by highest level of education completed.

Table 3

Age

Percent

20 or under

4.5%

21-30

6.1%

31-40

13.3%

41-50

18.8%

51-60

28.1%

61-70

20.6%

71 or over

5.9%

Table 4

Education

Percent

Less than HS

0.9%

Some HS

4.2%

HS diploma/equiv

6.3%

Some college, no degree

18.6%

Postsec non-degree award

2.9%

Associate’s

8.8%

Bachelor’s

25.1%

Master’s

17.0%

Doctoral/Professional

13.2%

•

Chart 1 shows the distribution of female respondents by age group and educational attainment.

art 1 Female Respondents by Age and Education

20 or

under

21-30 31-40 41-50 51-60 61-70 71 or

over

Count

Age

Less than HS

Some HS

HS diploma or equiv

Some college, no degree

Postsec non-degree award

Associate's

Bachelor's

Master's

Doctoral/Professional

•

Cha

rt 2 shows the distribution of male respondents by age group and educational attainment.

Chart 2 Male Respondents by Age and Education

20 or

under

21-30 31-40 41-50 51-60 61-70 71 or over

Count

Age

Less than HS

Some HS

HS diploma or equiv

Some college, no degree

Postsec non-degree award

Associate's

Bachelor's

Master's

Doctoral/Professional

•

The majority (76%) of the respondents were female. Individuals ranging in age from 51 to 60 years old

and having attained a bachelor’s degree represented the largest group of respondents, approximately

28% and 25% respectively. The largest number (n=58) of female respondents were 51-60 years of age

and had attained a bachelor’s degree. The second largest (n=50) number of female respondents were

also 51-60 years of age and had an educational attainment of some college but no degree. The largest

number (n=16) of male respondents were 61-70 years of age and had attained a doctoral or professional

degree. The second largest number (n=15) of male respondents were 51-60 years of age and had

attained a bachelor’s degree.

Age and Contact Method

The table below shows the number of respondents by age group who contacted/obtained information from

GARD by one of five methods. The data reflect only respondents who indicated both a contact method and

age.

Table 5

Contact Method

Age Group

≤

21-30

31-40

41-50

51-60

61-70

≥

Q&A only

Online contact form only

Email only

Phone only

Letter only

Table 6 shows the number of respondents by age group who indicated that they reviewed a Q&A on the

website and also contacted GARD by another method. The data reflect only respondents who indicated both

a contact method and age.

Table 6

Contact Method

Age Group

≤

21-30

31-40

41-50

51-60

61-70

≥

Q&A + online contact form

Q&A + email

Q&A + phone

Q&A + letter

Note: Table 5 does show that respondents between the ages of 51 and 60 were more likely to obtain information

from GARD by reviewing a Q&A, submitting the online contact form, or sending an email. As shown in Table

3, there were more survey respondents (~28%) in the 51-60 age group than in the other age groups. But, there is

not enough data to conclude for certain that a particular age group is more likely than another age group of

individuals to contact GARD using one particular method over another method.

Ethnicity and Race

86.9% of respondents indicated that they were not Hispanic or Latino and 7.6% indicated that they were

Hispanic or Latino.

Respondents were asked to indicate their race. The responses were as follows:

Table 7

Race

Percent

White

86.1%

Asian

4.9%

Black or African American

4.4%

American Indian or Alaska Native

3.4%

Native Hawaiian or Other Pacific Islander

1.2%

•

The majority of respondents (~87%) indicated their ethnicity as not Hispanic or Latino and their race as

White (~86%).

Who is being served by the GARD Information Center?

A snapshot of survey respondents:

Patient or a family member or friend of a patient

Female

Age 51-60

Bachelor’s degree

Not Hispanic or Latino

White

Prefer to speak and read in English

Reside in a suburban area

If reside in the U.S., reside in the southern region

If reside outside of the U.S., reside in Europe

Survey Objective 2. Identify how individuals learn about the GARD Information Center

Survey questions 1, 2 and 3 were designed to provide information about how respondents learned about the

Information Center.

Visits to GARD

Approximately 67% of survey respondents did not visit the Information Center in 2013. Of the survey

respondents who visited the Center in 2013, approximately 10% visited one time.

67% of survey respondents visited the Center one time in 2014 (includes visit in which respondent was

asked to complete the survey). Approximately 11% of survey respondents indicated that they visited the

Center two times in 2014 (between January 1, 2014 and April 25, 2014).

Note

of comparison: During the data collection period (as a part of GARD’s routine data collection), GARD customers

were asked if they had contacted GARD before. Of the 907 inquiries (excludes web page visitors), 390 stated no, 40 stated

yes, and 477 were not asked.

Find Out About the GARD Information Center

Survey respondents identified how they first found out about the Information Center. The responses are shown

in Table 8.

The majority of survey respondents did not seek information from the GARD Information Center in

2013 and as of April 2014 the majority of survey respondents had visited the Center one time.

Table 8

Resource

Percent

Search engine (e.g., Google, Yahoo, etc.)

64.3%

Other website

11.5%

Office of Rare Diseases Research website

8.5%

Other

4.2%

National Human Genome Research Institute website

3.0%

An organization

2.0%

Health care provider

1.5%

Friend/co-worker/colleague

1.4%

Media (e.g., magazine, newspaper, TV, radio)

1.3%

Family member

1.2%

Conference/meeting

0.1%

•

Other websites identified by at least 5 survey respondents were: National Library of Medicine/MedlinePlus

(n=28), Genetics Home Reference (n=13), Facebook group (n=12), National Organization for Rare

Disorders (n=10), National Institutes of Health main page (n=9), and WebMD (n=5).

Resources identified under Other that had not already been identified in other websites or organizations are

Twitter and teacher.

Only one conference/meeting was identified, and it was Rare Disease Day 2013.

Organizations identified include: American Thoracic Society, American Porphyria Foundation, Myositis

Association, Muscular Dystrophy Association, Charcot-Marie-Tooth Association, National Organization for

Rare Disorders, Multiple System Atrophy Coalition, American Lung Association, Center for Orphan

Disease Research & Therapy, Centers for Disease Control and Prevention, and Johns Hopkins University.

•

The majority of survey respondents found out about the Information Center by conducting a search

using a search engine such as Google or Yahoo. Approximately 11% of the survey respondents found

out about the Center by visiting the NCATS/ORDR or NHGRI website. The same percent (~11%) of

survey respondents found out about the Center by visiting other websites. If respondents did not find out

about the Center by using a search engine, they found out about it by visiting the NIH website (main

page, individual Institute pages, and National Library of Medicine, and Genetics Home Reference.

How do individuals find out about the GARD Information Center and how often do they visit GARD?

A snapshot of survey respondents:

By searching via search engine (Google, Yahoo)

Did not seek information from the Center in 2013

Did seek information from the Center one time between January and April 2014

Survey Objective 3, 4, and 5. Identify the reasons why individuals are seeking information from the GARD

Information Center, the type of information being sought, and where else customers may have sought

information

Survey questions 5-9 address survey objectives 3, 4, and 5.

Table 9 identifies the type of information that survey respondents were seeking from the Center.

Table 9

Type of Information

Percent

Specific disease or condition

59.4%

Treatment/medical care

38.7%

Diagnosis for symptoms

36.6%

Research study/clinical trial

27.3%

Clinical/genetic testing

18.9%

Alternative therapies

18.8%

Other

14.0%

Physician/specialist information

10.6%

Genetic services

10.2%

Lay advocacy/patient support organization

7.0%

Insurance

2.9%

Newborn screening

2.7%

Legal/financial

2.6%

• Respondents who indicated ‘other’ identified the specific information that they were seeking. The majority

of the ‘other’ responses could be categorized under specific disease or condition.

•

Specific disease/condition, treatment/medical care, diagnosis for symptoms, research study/clinical trial,

and clinical/genetic testing were the top five reasons that survey respondents were seeking information

from the Information Center. At least 90% of survey respondents were not seeking legal/financial,

newborn screening, insurance, and lay advocacy/patient support organization information.

Note of comparison: In 2011 the top 5 reasons for an inquiry were: (1) specific disease or condition, (2) treatment/medical

care, (3) physician/specialist information, (4) diagnosis of symptoms, and (5) research study/clinical trial.

71% of survey respondents indicated that they tried to or did obtain information from another source before

seeking information from the GARD Information Center. Where the survey respondents received their

information is identified in Table 10.

Table 10

Location of Information

Percent

Website

53.3%

Physician, nurse, other health care provider

22.7%

Organization

12.4%

Other

8.1%

Family member/friend/colleague

7.6%

Genetic counselor

2.9%

The websites, organizations, and ‘other’ specified by survey respondents were similar to the responses

received when survey respondents were asked how they found out about the Information Center (e.g.,

disease-specific organizations, various government agencies, Google, Mayo Clinic, NORD, WebMD,

Wikipedia).

Survey respondents who obtained information from another source before contacting the Center were asked

how satisfied they were with the information they received from the other sources. The responses were as

follows:

Not at all satisfied – 9.1%

Somewhat dissatisfied – 14.5%

Neither satisfied nor dissatisfied – 16.6%

Somewhat satisfied – 44.2%

Completely satisfied – 14.0%

•

These same survey respondents (i.e., those who obtained information elsewhere before contacting the

Center) were asked why they sought information from the Center after obtaining information elsewhere.

The responses were as follows:

For more in-depth information about a specific topic – 55.5%

For more information overall – 55.2%

For different information than I already had – 35.5%

Other – 11.3%

To talk to an information specialist – 8.2%

The responses provided by survey respondents for ‘other’ could be categorized into (1) for more in-depth

information about a specific topic, (2) for more information overall, and (3) for different information than I

already had.

In addition, some survey respondents specified that they sought information from the Center after seeking

information from other sources because they felt that the Center’s web pages/information (as a government

site) would be accurate and reliable.

•

The majority of survey respondents sought information from other sources before seeking

information from the GARD Information Center. They sought information primarily from a wide

variety of websites, such as disease-specific organizations, federal government agencies, and general

medical sites. Those seeking information from other sources were overall satisfied with the

information they received, but did contact the Center for more in-depth information about a specific

topic and to obtain more information overall.

What types of information are individuals seeking from the GARD Information Center and what other sources

were used to obtain the information?

A snapshot of survey respondents:

Top four types of information being sought: specific disease/condition, treatment/medical care, diagnosis for

symptoms, research study/clinical trial.

Bottom four types of information being sought: legal/financial, newborn screening, insurance, lay

advocacy/patient support organization.

Majority did obtain information from another source before contacting the Center.

Majority sought information from a website other than the Center’s web pages.

Majority of survey respondents indicated some level of satisfaction with the information they received from

other sources.

The reasons for contacting the Center after seeking information elsewhere were to obtain more in-depth

information about a specific topic and more information overall.

Survey Objective 6. Assess customer’s satisfaction with the GARD Information Center’s services (i.e.,

responses provided by information specialists and information available on the Center’s web pages)

Survey questions 10-20 address customer satisfaction.

Satisfaction with Information from Disease-Specific Information Pages (Q&A)

• Approximately 33% of the survey respondents indicated that they reviewed a Q&A on the Center’s web

pages to get the information they were looking for.

Below is what those respondents thought of the information they were provided in a Q&A.

•

The information…

Not at all

Somewhat

Very Much

Completely

answered my question(s)

9.0%

43.2%

28.4%

14.5%

gave me new information

13.9%

25.2%

36.1%

20.0%

gave me useful information

5.2%

25.5%

40.3%

24.2%

gave me information I could trust

3.9%

17.4%

41.3%

30.0%

met my needs

12.3%

35.2%

30.0%

15.5%

was right for my personal situation

11.6%

34.5%

29.4%

19.0%

was easy to read

2.6%

10.6%

43.2%

38.1%

was easy to understand

2.3%

11.9%

41.6%

37.7%

had the right amount of information

13.5%

31.9%

27.4%

20.3%

was easy to go through

2.6%

19.4%

37.4%

35.2%

78% of survey respondents indicated that they did not contact the Center (via online contact form, email,

phone, or letter) after reviewing a Q&A.

Satisfaction with Information Received by Email, Letter, or TTY

Of the survey respondents who did not receive information from the Center by reviewing a Q&A, 14.5%

received information by email after submitting the online contact form, 4.5% received information by email

after submitting a general email to the Center, 1.0% received information by letter (regular mail or fax), and

0.3% received information by TTY.

Below is what those respondents thought of the information they were provided.

The information…

Not at all

Somewhat

Very Much

Completely

answered my question(s)

10.9%

26.8%

31.1%

22.4%

gave me new information

13.1%

25.1%

28.4%

23.5%

gave me useful information

9.8%

21.3%

33.9%

24.0%

gave information I could trust

7.7%

16.4%

31.7%

met my needs

14.8%

20.2%

27.9%

24.0%

was right for my personal situation

15.3%

19.7%

29.0%

24.6%

was easy to read

6.0%

14.8%

32.2%

35.0%

was easy to understand

5.5%

14.8%

31.7%

37.7%

had the right amount of information

13.7%

22.4%

25.1%

26.8%

was easy to go through

4.9%

14.2%

34.4%

35.0%

came within a reasonable time

9.8%

15.8%

33.9%

29.0%

One-third of survey respondents reviewed a Q&A. The majority of those survey respondents did not

contact the Center again after reviewing a Q&A.

Overall, these survey respondents felt that the Q&A provided them with new information, useful

information, and information they could trust. In addition, they felt that the Q&A was easy to read,

understand and go through.

•

Approximately 65% of survey respondents did not review a Q&A. Of those survey respondents, 19%

received information from the Center by email (after submitting the online form or a general email)

and 1.0% received information by regular mail or fax.

Overall, these survey respondents indicated a high level of satisfaction (by responding very much or

completely to the respective question) with the information they received from the GARD

Information Center.

Satisfaction with Information from an Information Specialist by Phone

• Of the survey respondents who did not receive information from the Center by reviewing a Q&A, 4.0%

received information by speaking to one of the Center’s Information Specialist. Below is what those

respondents thought of the information they were provided.

The information specialist…

Not at all

Somewhat

Very Much

Completely

answered my question(s)

3.0%

18.2%

39.4%

36.4%

gave new information

15.2%

21.2%

27.3%

33.3%

gave useful information

3.0%

27.3%

24.2%

42.4%

gave information I could trust

3.0%

15.2%

24.2%

51.5%

gave information that met my needs

9.1%

24.2%

36.4%

gave information right for my personal situation

12.1%

18.2%

21.2%

39.4%

gave information that was easy to understand

3.0%

9.1%

24.2%

57.6%

gave the right amount of information

12.1%

18.2%

36.4%

27.3%

was informative

6.1%

18.2%

27.3%

39.4%

was sensitive to my needs

3.0%

6.1%

33.3%

51.5%

The survey respondents who received information from the Center by speaking with an information

specialist indicated a high level of satisfaction (by responding very much or completely to the

respective question) with the information they received from the information specialist.

What Will Respondents Do With the Information?

All survey participants were asked what they did or plan to do with the information they received from the

Center. The responses were:

Action

Percent

Review it, but do not do anything else with it.

20.5%

Make contact with individuals/organizations identified.

18.8%

Share information with health care providers.

29.6%

Share information with patients.

5.9%

Share information with family members or friends.

33.1%

Other

18.6%

Responses given for ‘other’ can be categorized as follows:

Self-edification (i.e., increase own knowledge) – 22.5%

Educational purpose (e.g., presentation, report/paper, school assignment) – 21.4%

Share with others (some of these responses fit in one of the response categories such as share with

family member, patient, health care provided, friend) – 13.3%

Going to do nothing (i.e., did not find what looking for, information not helpful) – 6.4%

Patient care or management – 5.8%

Additional followup/research – 2.3%

•

Based on the answers to the question response categories and the responses given for the ‘other’

category, survey respondents shared the information with others (e.g., health care providers, family

members, colleagues, etc.) or obtained the information for personal use (i.e., self-edification or

educational).

Overall Satisfaction with the GARD Information Center

Survey respondents were asked to indicate how satisfied they were with the GARD Information Center.

Responses were:

Satisfaction

Percent

Not at all satisfied

3.8%

Somewhat dissatisfied

6.6%

Neither satisfied nor dissatisfied

20.4%

Somewhat satisfied

35.4%

Completely satisfied

26.9%

Overall, the survey respondents were somewhat to completely satisfied with the GARD Information

Center.

Use of the GARD Information Center Again

Approximately 48% of survey respondents indicated that they were likely to use the Center again, and

approximately 39% would definitely use it again.

The survey respondents would likely contact the Information Center again if they had a question

related to a genetic and rare disease.

Assessment of satisfaction with GARD Information Center services

A snapshot of survey respondents:

There were more survey respondents (~65%) who received information from the Center by email or phone

as compared to survey respondents (~33%) who obtained information from the Center by reviewing a Q&A

on the GARD web page.

There were more survey respondents (19%) who obtained information by email (after submitting the online

form or general email) as compared to survey respondents (4%) who obtained information by speaking with

an information specialist.

Of the survey respondents who reviewed a Q&A, the majority (78%) did not seek further information from

the Center.

Using the responses to the questions about the effectiveness of the information received from the Center

(e.g., usefulness, personal applicability, timeliness, understandability, and appropriateness) as a proxy for

satisfaction, the survey respondents who were most satisfied were those who obtained information by

speaking with an information specialist. In other words, the more personal and open-ended the interaction

was between inquirers and GARD, the more satisfied they were.

•

The survey respondents who were least satisfied were those who indicated that they reviewed a Q&A to get

the information they were looking for.

Overall, survey respondents were somewhat satisfied (as indicated by 35.4% of respondents) with the

GARD Information Center.

Survey respondents are most likely to share the information with others (e.g., provider, family member, co-

worker, etc.) or use the information to educate themselves or for educational purposes (e.g., report,

presentation, etc.).

Less than half (48%) of survey respondents would likely contact the Center again.

Individual Suggestions for Improving Information Received from the GARD Information Center and Additional

Comments about the Information Center

Four hundred and forty-one (441) survey respondents provided suggestions and comments for improving the

GARD Information Center.

Individual suggestions for improving the information available from the Center and suggestions for improving

the Center overall were categorized. Even though the comments were from individuals, they are worth

reviewing carefully and developing a plan to address them where possible. The individual comments are located

in Appendix E, and the categories are listed below.

•

Use less technical or difficult language/terms

Improve the accuracy of the information

Update the information

Provide information about a particular disease or topic (i.e., information currently not provided or can’t be

found on the website)

Provide more information about a particular disease or topic (i.e., not enough information currently

provided on the website)

Provide or link to other types of information (or additional information)

Modify structure/format of response

Modify website capabilities/features

Improve website navigation

Improve response time

Conduct outreach/advertise

Improve communication (written and oral) with GARD customers

Other suggestions/comments

Included in the 441 suggestions/comments for improvement were many comments related to the high level of

satisfaction with the information, the Center, and the information specialists. These comments are located in

Appendix F.

GARD Information Center Customer Satisfaction Survey Findings – 2006 vs. 2014

In 2006 a customer satisfaction survey of GARD was conducted. Although the survey methodology and

questions where reflective of how GARD provided its services at the time, some questions from the 2006

survey were asked in the 2014 survey. A comparison of relevant items is shown in the table below.

Descriptor

Year of User Satisfaction Survey

2006

2014

Primary Gender

Female

Age of Inquirers

31-40

51-60

Education

Post-graduate

Bachelor’s

Ethnicity

Non-Hispanic or Latino

Self-described Race

White

Preferred Language

English

Role (top 2 roles)

Patient, Family member/friend of patient

Residence

U.S. – Southern & Western states

U.S. – Southern states

Visits to GARD (in year of

survey)

One

Find GARD

Internet search engine (Google, Yahoo)

Inquiry Reason (top 2

reasons)

Specific disease/condition,

treatment/medical care

Specific disease/condition,

treatment/medical care

Research before contacting

GARD

Yes

Plan for information

Share with others

Use GARD again

Definitely

Likely

Limitations

Every method of scientific inquiry is subject to limitations and has its advantages and disadvantages. As such,

the use of survey research and, in particular, the use of a web-based survey to obtain information about

customer satisfaction with GARD had its limitations, advantages, and disadvantages.

A limitation of this and the previous user satisfaction survey was the use of a convenience sample. Because of

the use of a convenience sample for this survey, the data and conclusions drawn from the data are only

representative of the 930 individuals who contacted GARD for information and choose to complete a survey.

Conclusions drawn from this evaluation do not necessarily represent the larger population of individuals who

contact GARD for information.

Convenience samples are probably the most often used sampling method, because subjects are easily accessible

and data collection is relatively fast, inexpensive, and easy. Use of a convenience sample was chosen as the

method of sampling for this survey, because the population of GARD customers is too large and it would have

been impossible to include every individual who contacts GARD. Due to the personal and often sensitive nature

of requests it is important to assure and respect customer confidentiality and privacy; therefore, a database of

GARD customers does not exist. If such a database did exist, it would have been possible to use a random

sampling method.

Another limitation of the most recent survey was insufficient pretest data. OMB regulations limit the pretest

sample to nine individuals, but the author and ICF were only able to gather data from five individuals who

contacted GARD during the pretest timeframe. Had there been sufficient pretest data, it might have become

obvious to detect the confusion with the data collection system and make changes to avoid the confusion that

was subsequently detected in the final survey.

Additionally, the estimated OMB-approved sample size was very rapidly reached. Therefore, there was no

interim data set available. After the survey closed and the data were analyzed, it was noted that some

respondents completed the survey before reviewing the information from GARD. Without an interim data set to

review and analyze, survey and/or data collection problems were not detected and changes in the collection

approach were not undertaken. Completion of the survey without reviewing the information sought from GARD

may have created bias and slanted the responses in a negative direction from some of the survey respondents.

There are disadvantages to using web-based surveys in that they tend to be restricted to individuals who have

access to a computer and Internet, are keyboard and Internet literate, and understand what is required of them in

taking a web-based survey. Age can be a contributing factor to less familiarity and comfort with using the

Internet and taking web-based surveys. Given that the largest group of respondents were between the ages of 51

and 60, age may have been a contributing factor for some respondents’ lack of understanding that the survey

pop-up box did not require them to take the survey immediately (i.e., prior to their review of the information

from GARD).

Counter to the disadvantages, there are many benefits to using web-based surveys including: inclusiveness

(being able to reach a global audience), relatively inexpensive (after initial set-up), and faster data analysis (data

already in electronic format). A web-based approach was chosen for this survey for the many benefits.

Additionally, to be inclusive of individuals who contacted GARD by phone and who may not have otherwise

chosen to complete a web-based survey, the option was available to complete the survey by phone in which

case the interviewer entered the responses in the web-based survey structure.

Recommendations

The recommendations below stem from feedback provided by survey respondents. The recommendations can

be used as a springboard to explore different methods of obtaining more detailed information about the

experience individuals have with the GARD Information Center and to discuss ways to improve the service.

Conduct Ongoing Data Collection. Although many of the demographics were similar between the 2006

convenience sample and the 2014 convenience sample, the age and education of the survey respondents

changed. In 2014 more survey respondents were in the 51-60 age range and had attained a bachelor’s degree as

compared to most of the 2006 survey respondents who were in the 31-40 age range and had obtained a post-

graduate degree. To best serve all GARD customers in terms of how best to present information, it is important

to always have a profile of individuals who contact GARD.

It is recommended that GARD obtain the appropriate approvals to collect voluntarily provided, limited

demographic data from individuals who contact GARD using the online contact form or by phone. This data

should be reviewed on a regular basis and changes made, as needed, to the information service in response to

changes, if any, in the demographics of the population of GARD customers.

Conduct Outreach. Based on feedback from survey respondents, it would benefit the community at large and the

genetic and rare diseases community in particular if GARD were to increase its visibility. It first will need to be

determined if conducting outreach and advertising is a useful goal for GARD given available funding. If it is an

attainable goal, a plan should be developed that outlines the audiences GARD seeks to reach and how to reach

those audiences. It might be helpful to use focus groups to identify ways to reach specific audiences. Data from

this survey suggests that the following audiences are potential targets for any future outreach activities:

healthcare professionals, researchers, and educators. Additionally, GARD should make efforts to target

individuals with the following characteristics: live in rural or other remote communities, communicate in

languages other than English, and identify themselves as a minority. Efforts should also be made to reach

people with more limited education.

Improve Inquiry Responses. Overall the survey respondents were satisfied with the information they received

from GARD. Survey respondents provided a wealth of suggestions and comments to improve the information

that GARD provides. Below are the improvements that were suggested by survey respondents. Individual

suggestions and comments are located in the appendix of this report and should be reviewed by GARD staff and

NCATS project management to determine accuracy of the suggestion, feasibility for implementation, and

priority for change based on practical implementation issues.

•

Use less technical or difficult language and terms (especially if you want to reach an audience with more

limited educational attainment).

•

Improve the accuracy/timeliness of the information.

Expand the number of conditions for which information is available on the website.

Expand the information provided for some conditions.

Provide other types of information and links, for example, information for teachers of children with rare

disorders and photos to illustrate features of a particular syndrome.

Improve response time.

Prioritize recommendations for change based on practical implementation issues.

Improve Functionality of the GARD Web Pages. Some of the concerns expressed by survey respondents about

the functionality of the GARD web pages should be addressed in a work plan to be developed by GARD staff

following this survey. It is recommended that GARD staff review the suggestions and comments provided by

survey respondents and conduct usability testing prior to implementing changes to the GARD web pages.

Conduct a Review of Existing Resources. One way to implement the survey results would be to conduct a

review of other genetic and/or rare diseases information services or portals currently available to the public (i.e.,

similar to a review of the literature). The main purpose for the review would be to eliminate redundancy in the

service/information provided by GARD and to more clearly articulate what is unique, different, or of additional

value about seeking information from GARD as compared to seeking information from other sources. For

example, the ability to speak with an information specialist may be a unique or value-added service not

available from other sources. The information could also be used for outreach and marketing activities. For

example, if access to information specialists is unique to GARD, this value-added service could be emphasized

in outreach and marketing activities. Before entering any of the GARD web pages there should be a concise

description of what GARD is and is not (e.g., what services/information is and is not provided and what is

unique about GARD).

Conclusion

Although similarities have been noted between the 2014 survey results, the 2006 survey results, and the very

limited data collected routinely by GARD, a convenience sample was used for each of those data collection

opportunities and caution must be taken in making inferences from survey and trend data to the larger

population of GARD customers. Although the results from the 2014 survey cannot be generalized to the larger

population of GARD customers, the survey results are very useful and provide rich qualitative information. The

survey data yield insights into the needs of the survey respondents and their experiences using the GARD

Information Center.

It is the author’s opinion that the survey respondents were satisfied with the information they received from

GARD. The most satisfied respondents were those who received information by speaking with a GARD

information specialist. Speaking to an information specialist appeared to be a desirable way of obtaining

information especially for individuals who perhaps did not have access to a computer, or who may have felt

more comfortable expressing themselves to a person rather than in an email or letter, or who needed to clarify

the information they obtained from the GARD web pages. By offering a variety of ways (i.e., website, email,

phone, and letter) to obtain information, GARD has made information about genetic and rare diseases more

easily accessible to many individuals. For example, during the implementation of the survey in 2006 GARD

received an average of approximately 300 inquiries per month by regular email, phone, and letter. In 2008 the

GARD web pages were launched. Thus far in 2014 GARD has received an average of approximately 600

inquiries per month by online contact form, email, phone, and letter. There is also an average of approximately

300,000 to 400,000 website visitors per month (i.e., individuals visiting a Q&A or other GARD web page).

Based on the findings of this survey, it can be concluded that GARD is providing a valuable, useful, and unique

service to the genetic and rare diseases community. It is recommended that prior to the development of the next

GARD work plan, the GARD funding agencies and staff review the survey findings and the recommendations

in this report to determine which suggestions, recommendations, and activities are feasible and appropriate to

include in the next work and implementation plan.

Content Notes

Lisa Lanier, Ph.D. was previously employed by the National Human Genome Research Institute and served

as the Project Officer for the Genetic and Rare Diseases Information Center from 2002-2004. Dr. Lanier

developed the survey and performed the data analysis for the 2006 GARD customer satisfaction survey.

There were two respondents who indicated that they received information from GARD by TTY. These

responses must have been in error as usage data collected by GARD during the data collection period indicated

that there were no TTY calls/transmissions during the data collection period.

Not enough data was collected in the survey pretest to identify that Question 16 (i.e., what going to do with

information received) may have been confusing (based on the responses given for the ‘other’ category) to some

survey respondents, particularly those that only reviewed a Q&A. Those respondents may not have equated

‘received information from GARD’ with their action of reviewing a Q&A on the website. There were 24

responses pertaining to not receiving any information at all (or yet) or that the survey should have been held

until a respondent completed his/her review of the information. In addition, it became evident from the ‘other’

responses that the response categories needed to be modified (e.g., add categories, collapse categories, reword

categories).

It appears as if some visitors to the GARD web pages may have been confused when they saw the survey

pop-up in the corner of a GARD web page. They may not have realized that they could ignore the pop-up

window until they were ready to take the survey. Therefore, some survey respondents did complete the survey

before reviewing the information available on the Center’s web pages. There were over 50 comments pertaining

to not having time to review or not receiving information from the GARD Information Center before having to

complete the survey. Additionally, the responses from these respondents may indeed have skewed the results by

increasing the number of incomplete or negative responses to some of the survey questions.

Appendix A

English and Spanish Web-Based Customer Satisfaction Survey

//insert GARD survey banner//

[Landing Page]

National Institutes of Health, National Center for Advancing Translational Sciences - Office of Rare

Diseases Research and National Human Genome Research Institute

Genetic and Rare Diseases (GARD) Information Center

Customer Satisfaction Survey

Institutos Nacionales de la Salud, Centro Nacional para el Avance de las Ciencias Aplicadas (NCATS) – La Oficina de

Investigaciόn de Enfermedades Raras y El Instituto Nacional de Investigación del Genoma Humano

El Centro de Información sobre Enfermedades Genéticas y Raras (GARD por sus siglas en inglés)

su Servicio de Encuesta de Satisfacciόn del Cliente

BURDEN DISCLOSURE STATEMENT:

Public reporting burden for this collection of information is estimated to average 10 minutes per response, including the time for

reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the

collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of

information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other

aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705

Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0648). Do not return the completed form to this

address.

Cláusula de divulgación de la responsabilidad

Según los reportes del público para completar esta recopilación de información se estima un promedio de 10 minutos por respuesta,

incluyendo el tiempo para revisar las instrucciones, buscar las fuentes de datos existentes, compilar y guardar los datos necesarios y

completar y revisar la recopilación de información. Una agencia no tiene la obligación de realizar o de patrocinar una recopilación de

información y ningún individuo está obligado a responder a una recopilación de información, a menos que esta posea un número de

control válido de la Oficina de Gerencia y Presupuesto (Office of Management and Budget – OMB, por sus siglas). Envíe sus

comentarios sobre el estimado del promedio de tiempo necesario para las respuestas o sobre cualquier otro aspecto de esta

recopilación de información, incluyendo sugerencias para reducir la responsabilidad del público a: NIH, Project Clearance Branch,

6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0648). No envíe por el correo a esta dirección el

formulario completado.

//A

sk All//

//required response//

Lang_1. Please select your preferred language:

1. English

2. Spanish

Lang_1. Por favor seleccione su lenguaje preferido:

1. Inglés

2. Español

-------------//PAGE BREAK//

National Institutes of Health, National Center for Advancing Translational Sciences - Office of Rare

Diseases Research and National Human Genome Research Institute

Genetic and Rare Diseases (GARD) Information Center

Customer Satisfaction Survey

Institutos Nacionales de la Salud, Centro Nacional para el Avance de las Ciencias Aplicadas (NCATS) – La Oficina de

Investigaciόn de Enfermedades Raras y El Instituto Nacional de Investigacion del Genoma Humano

El Centro de Información sobre Enfermedades Genéticas y Raras (GARD por sus siglas en inglés)

su Servicio de Encuesta de Satisfacciόn del Cliente

hank you for taking the time to take part in this survey. Information from this survey will be used to help us

improve our inquiry responses to better meet your needs. No personal identifiers will be attached to your survey

responses. Your answers will be kept confidential.

Gracias por su participaciόn en la Encuesta de Satisfacción del Cliente. La informaciόn que obtuvimos en esta

encuesta nos ayudará a mejorar nuestro servicio para satisfacer mejor las necesidades de nuestro público. No hay

identificadores personales en sus respuestas a la encuesta. Sus respuestas serán confidenciales.

Instructions:

Please read the following instructions before beginning this survey.

Instrucciones:

Por favor lea las siguientes instrucciones antes de comenzar la encuesta.

MOVING FORWARD:

Click on the NEXT button located at the bottom of the page to save your responses and continue to the next page.

AVANZAR:

Clique en el botόn SIGUIENTE situado en la parte inferior de la página para salvar sus respuestas y continuar a la

siguiente página.

MOVING BACK:

Use the PREVIOUS button located at the bottom of the page to view your responses on a previous page. You may

change your answers to previously entered responses.

RETROCEDER:

Utilice el botόn ANTERIOR en la parte inferior de la página para ver sus respuestas en la página anterior. Usted puede

cambiar sus respuestas previas.

o not use the BACK button of your browser to return to the previous page. Using your browser’s BACK button

may cause you to exit the survey, and your responses will be lost.

No utilice el botόn de retorno de su navegador para regresar la página anterior. Utilizando el botόn de RETORNO

de su navegador puede resultar en que usted salga de la encuesta, y se perderán sus repuestas.

f you experience any questions about the GARD Information Center Customer Satisfaction Survey, please

contact the Help Desk at 1-888-205-2311 Monday-Friday, 9:00am – 8:00pm EST or email

the GARD Information

Center INFO@NIH.GOV //hyperlink//.

Si usted tiene alguna pregunta sobre esta Encuesta de Satisfacciόn del Cliente del Centro de Informaciόn GARD,

por favor póngase en contacto con el Servicio de Asistencia discando el número 1-888-205-2311 de lunes a viernes,

de 9:00 am – 8:00 pm EST o utilice el correo electrόnico del Centro de Información GARD

GARDINFO@NIH.GOV.

If you experience any technical issues accessing this survey, please contact a technical support specialist at 1-301-

407-6652.

Si usted tiene cualquier problema técnico al realizar esta encuesta, por favor pόngase en contacto con especialistas

de soporte técnico llamando el número 1-301-407-6652.

-----------//PAGE BREAK//

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q1. How did you first find out about the GARD Information Center? (select one)

1. Search engine (e.g., Google, Yahoo, etc.)

2. National Center for Advancing Translation Science (NCATS)/Office of Rare Diseases Research (ORDR)

website

3. National Human Genome Research Institute (NHGRI) website

4. Other website (specify)[Q1_web_other] ________________ // DO NOT FORCE SPECIFY RESPONSE//

5. Family member

6. Friend/co-worker/colleague

7. Health care provider

8. Conference/meeting (specify) [Q1_conf_other] ________________ // DO NOT FORCE SPECIFY

RESPONSE//

9. An organization (specify) [Q1_org_other] __________________ // DO NOT FORCE SPECIFY

RESPONSE//

10. Media (magazine, newspaper, television, radio)

11. Other (specify) [Q1_other] ________________ // FORCE SPECIFY RESPONSE//

Q1. ¿Cómo usted inicialmente se enteró del Centro de Información GARD? (seleccione una

opción)

1. Mecanismos de búsqueda de la red (e.g., Google, Yahoo, etc.)

2. Centro Nacional para el Avance de las Ciencias Aplicadas (NCATS) – La Oficina de

Investigaciόn de Enfermedades Raras (ORDR)

3. Sitio en la Red del Instituto Nacional de Investigación del Genoma Humano (NHGRI)

4. Otros sitios de la red (especifique)

5. Parientes

6. Amigos o colegas de trabajo

7. Proveedor de servicios de salud

8. Congresos o Reuniones (especifique)

9. Organizaciones (especifique)

10. Medios de comunicación (revistas, periódicos, televisión, radio)

11. Otros (especifique)

//A

sk all//

//*88 = NO ANSWER (HIDDEN)//

//RANGE 0-1000//

Q2. How many times did you use the GARD Information Center to obtain information in 2013? //REQUIRE WHOLE

NUMBER DURING VALIDATION; Display text “Please provide a whole number”//

Q2. ¿Cuántas veces utilizó el Centro de Información GARD para obtener información en el año 2013?

//Ask all//

//*88 = NO ANSWER (HIDDEN)//

//RANGE 0-1000//

Q3. How many times have you used the GARD Information Center to obtain information in 2014, including your most

recent effort? //REQUIRE WHOLE NUMBER DURING VALIDATION; Display text “Please provide a whole

number”//

Q3. ¿Cuántas veces ha utilizado el Centro de Información GARD usted para obtener información en el año 2014,

incluyendo su intento más reciente?

-----------//PAGE BREAK//

//Ask all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

The following questions refer to your most recent interaction with the GARD Information Center (e.g. Q&A

website, contact with the GARD Information Center information specialist through email, telephone, fax, etc.).

Q4. Which of the following best describes you during your most recent effort to obtain information from the GARD

Information Center? (select one)

1. Patient

2. Family or friend of patient

3. Physician

4. Nurse

5. Genetic Counselor

6. Other health care provider or allied health professional (specify) [Q4_provider_oth] ________________// DO

NOT FORCE SPECIFY RESPONSE//

7. Researcher/Scientist

8. Journalist/Media professional

9. Teacher or Professor/Educator

10. Student

11. Other (specify) [Q4_other] ________________// FORCE SPECIFY RESPONSE//

Las siguientes preguntas se refieren a su experiencia más reciente con el Centro de

Informaciόn GARD (por ejemplo, Q&A, la página en la red de preguntas y respuestas, o

contacto con un especialista en informaciόn del Centro de Información GARD través del

correo electrónico, teléfono, fax, etc.)

Q4. ¿Con cuál de las siguientes opciones usted se identifica mejor como usuario en su experiencia

más reciente para obtener información del Centro de información GARD? (seleccione una opción)

1. Paciente

2. Pariente o amigo del paciente

3. Médicos

4. Enfermero (a)

5. Consejero (a) de Genética

6. Otro tipo de proveedor de la salud o de profesional de la salud (especifique)

7. Investigador/Científico

8. Periodista/Profesional de Medios de Comunicación

9. Maestro o Profesor/Educador

10. Estudiante

11. Otro (especifique)

//Ask all//

//multiple response//

//*88 = NO ANSWER (HIDDEN)//

Q5. What type of information were you trying to obtain? (select all that apply)

1. Alternative therapies

2. Clinical/genetic testings

3. Diagnosis for symptoms

4. Genetic services

5. Insurance

6. Lay advocacy group/Patient support organization

7. Legal/financial

8. Newborn screening

9. Physician/Specialist information

10. Research study/clinical trial

11. Specific disease or condition

12. Treatment/medical care

13. Other (specify) [Q5_other] ________________// FORCE SPECIFY RESPONSE//

Q5. ¿Qué tipo de información estaba tratando de obtener usted? (seleccione todas las opciones que se

apliquen)

1. Tratamientos alternativos

2. Exámenes Clínicos/Genéticos

3. Diagnóstico de los síntomas

4. Servicios de genética

5. Seguro

6. Organizaciones de defensa de pacientes/Grupos de Apoyo

7. Legal/Financiera

8. Pruebas de detecciόn para recién nascidos

9. Remisiones para medicos o especialistas

10. Estudios de investigación/ Estudios clínicos

11. Información específica acerca una enfermedad o condición

12. Tratamiento/Cuidados médicos

13. Otros (especifique)

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q6. Did you try to obtain or obtain information from another source before using the GARD Information Center? (select

one)

1. Yes

2. No

Q6. ¿Usted obtuvo o trató de obtener información de otra fuente antes de utilizar el Centro de Información GARD?

(seleccione una opción)

1. Si

2. No

----

---------//PAGE BREAK//

//Ask if Q6 = 1 or Q6 = No Answer//

//multiple response//

//*88 = NO ANSWER (HIDDEN)//

Q7. Where did you do your research/get your information before using the GARD Information Center? (select all that

apply)

1. Website (specify) [Q7_web_oth] ________________// DO NOT FORCE SPECIFY RESPONSE//

2. Organization (specify) [Q7_org_oth] ________________// DO NOT FORCE SPECIFY RESPONSE//

3. Physician, nurse, or other health care provider/professional

4. Genetic counselor

5. Family member/friend/co-worker/colleague

6. Other (specify) [Q7_other] ________________// FORCE SPECIFY RESPONSE//

Q7. ¿Qué otros servicios usted utilizó antes de utilizar el Centro de Información GARD?

(seleccione todas las que apliquen)

1. Sitios en la red (especifique)

2. Organizaciones (especifique)

3. Médicos, enfermeras u otros profesionales o proveedores de salud

4. Consejero (a) genético

5. Pariente/Amigo/Colega

6. Otros (especifique)

//A

sk if Q6 = 1 or Q6 = No Answer//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q8. Overall, how satisfied were you with the information you received from those sources? (select one)

1. Not at all satisfied

2. Somewhat dissatisfied

3. Neither satisfied nor dissatisfied

4. Somewhat satisfied

5. Completely satisfied

Q8. De forma general, ¿cómo cataloga su nivel de satisfacción con la información que ha recibido

de esas fuentes? (seleccione una opción)

1. Completamente insatisfecho

2. Más o menos insatisfecho

3. Indiferente

4. Más o menos satisfecho

5. Completamente satisfecho

//A

sk if Q6 = 1 or Q6 = No Answer //

//multiple response//

//*88 = NO ANSWER (HIDDEN)//

Q9. Why did you use the GARD Information Center after researching your question/getting information from other

sources? (select all that apply)

1. For more information

2. For more in-depth information about a specific topic (e.g., treatment options, testing centers)

3. For different information than I already had

4. To talk to an information specialist

5. Other (specify) [Q9_other] ________________// FORCE SPECIFY RESPONSE//

Q9. ¿Por qué usted decidió utilizar el Centro de Información GARD después de investigar/

obtener información de otras fuentes? (seleccione todas las opciones que se apliquen)

1. Para obtener más información

2. Para obtener información más detalladas sobre un asunto específico (ej., opciones de

tratamiento, centros de exámenes diagnósticos)

3. Para obtener información sobre otros asuntos

4. Para hablar o ponerme en contacto con un especialista en información

5. Otros (especifique)

-----------//PAGE BREAK//

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q10. Did you review a Q&A on the GARD website to get information you were looking for? (select one)

1. Yes

2. No

Q10. ¿Usted revisó las páginas de preguntas y respuestas del sitio en la red del Centro de Información GARD para

conseguir la información que buscaba? (seleccione una opción)

1. Si

2. No

-----------//PAGE BREAK//

/Ask if Q10 = 2 or Q10 = No Answer //

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q11. How did you get the information you were looking for from the GARD Information Center? (select one)

1. I received a response by email after submitting the online email form located on the GARD website. (go to Q14)

2. I received a response by email after submitting a general email to GARD. (go to Q14)

3. I spoke to a GARD information specialist. (go to Q15)

4. I received a letter by regular mail or fax. (go to Q14)

5. I received a response by TTY (telecommunication device for the deaf). (go to Q14)

Q11. ¿Cómo usted obtuvo la información que estaba buscando en el Centro de Información GARD? (seleccione una

opción)

1. Recibí la respuesta a través de correo electrónico después de enviar una sumisión de correo electrónico localizada

en el sitio de GARD en la red.

2. Recibí una respuesta a través de correo electrónico después de enviar un correo electrónico a GARD.

3. Hable con un especialista en información de GARD.

4. Recibí una carta a través del correo regular o de fax.

5. Recibí una respuesta a través de TTY (teletipo, un dispositivo utilizado para ayudar a las personas sordas a

comunicarse).

-----------//PAGE BREAK//

//Ask if Q10 = 1//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q12. Please tell us what you think about the information you were provided on the GARD Information Center Q&A.

(select one answer for each statement)

The information…

Not at all Somewhat Very much Completely

1. …answered my question(s). 1 2 3 4

2. …gave me new information. 1 2 3 4

3. …gave me useful information. 1 2 3 4

4. …gave me information I could trust. 1 2 3 4

5. …met my needs. 1 2 3 4

6. …was right for my personal situation. 1 2 3 4

7. …was easy for me to read. 1 2 3 4

8. …was easy to understand. 1 2 3 4

9. …had the right amount of information. 1 2 3 4

10. …was easy to go through. 1 2 3 4

Q12. Por favor, díganos lo que piensa acerca de la información que fue proporcionada en las páginas

de Preguntas y Respuestas Q&A del Centro de Información GARD. (seleccione una respuesta para

cada afirmación)

La información...

En desacuerdo Ni de acuerdo ni en desacuerdo Muy de acuerdo Totalmente de acuerdo

1. ...respondió mi (s) pregunta (s).

2. ...me proporcionό información adicional.

3. ...me proporcionó información útil.

4. ...me proporcionó información confiable.

5. ...me proporcionó lo que necesitaba.

6. ...fue adecuada a mi situación personal.

7. ...fue fácil de leer.

8. ...fue fácil de entender.

9. ...tenía información suficiente.

10. ...fue fácil de navegar.

11. …fue enviada dentro de un límite de tiempo razonable

Ask if Q10 = 1//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q13. Did you contact the GARD Information Center in addition to reviewing a Q&A? (select one)

1. No

2. Yes, I submitted the online email form located on the GARD Information Center website.

3. Yes, I sent a general email.

4. Yes, I called and spoke to a GARD information specialist.

5. Yes, I sent a letter by regular mail or fax.

Q13. ¿Usted se comunicó con el Centro de Información GARD, además de leer las Preguntas y

Respuestas Q&A? (seleccione una opción)

1. No

2. Si, yo sometí un formulario de correo electrónico localizado en el sitio del Centro de

Información de GARD.

3. Si, yo envié un correo electrónico.

4. Si, yo llame y hable con un especialista en información del Centro de Información GARD.

5. Si, yo envié una carta a través del correo regular o envié un fax.

-----------//PAGE BREAK//

/Ask if Q11 = (1,2,4, 5) or if Q13 = (2,3,5)//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q14. Please tell us what you think about the information you received in the response. (select one answer for each

statement)

The information…

Not at all Somewhat Very much Completely

1. …answered my question(s). 1 2 3 4

2. …gave me new information. 1 2 3 4

3. …gave me useful information. 1 2 3 4

4. …gave me information I could trust. 1 2 3 4

5. …met my needs. 1 2 3 4

6. …was right for my personal situation. 1 2 3 4

7. …was easy for me to read. 1 2 3 4

8. …was easy to understand. 1 2 3 4

9. …had the right amount of information. 1 2 3 4

10. …was easy to go through. 1 2 3 4

11. …came within a reasonable time. 1 2 3 4

Q14. Por favor, díganos lo que piensa acerca de la información que ha recibido en la respuesta.

(seleccione una respuesta para cada afirmación)

La información...

En desacuerdo Ni de acuerdo ni en desacuerdo Muy de acuerdo Totalmente de acuerdo

. ...respondió mi (s) pregunta (s).

. ...me proporcionό información adicional.

. ...me proporcionó información útil.

. ...me proporcionó información confiable.

. ...me proporcionó lo que necesitaba.

. ...fue adecuada a mi situación personal.

. ...fue fácil de leer.

. ...fue fácil de entender.

. ...tenía información suficiente.

0. ...fue fácil de navegar.

1. …fue enviada dentro de un límite de tiempo razonab

1 e.

----

---------//PAGE BREAK//

//Ask if Q11 = 3 or if Q13 = 4//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q15. Please tell us what you think about the information you received over the phone. (select one answer for each

statement)

The information specialist…

Not at all Somewhat Very much Completely

1. …answered my question(s). 1 2 3 4

2. …gave new information. 1 2 3 4

3. …gave useful information. 1 2 3 4

4. …gave information I could trust. 1 2 3 4

5. …gave information that met my needs. 1 2 3 4

6. …gave information right for my personal situation. 1 2 3 4

7. …gave information that was easy to understand. 1 2 3 4

8. …gave the right amount of information. 1 2 3 4

9. …was informative. 1 2 3 4

10. …was sensitive to my needs. 1 2 3 4

Q15. Por favor, díganos lo que piensa de la información que recibió por teléfono. (seleccione una

respuesta para cada afirmación)

El especialista en información...

En desacuerdo Ni de acuerdo ni en desacuerdo Muy de acuerdo Totalmente de acuerdo

1. …respondió mi (s) pregunta (s)

2. …me proporcionó informaciónes adicionales.

3. …me proporcionó información útil.

4. …me proporcionó información confiable.

5. ...me proporcionó lo que necesitaba.

6. ...fue adecuada a mi situación personal.

7. …me proporcionó información fácil de entender.

8. …me proporcionó una cantidad suficiente de información.

9. ...era informativa.

10. …fue colocada de forma delicada y sensible.

-----------//PAGE BREAK//

//Ask all//

//multiple response//

//*88 = NO ANSWER (HIDDEN)//

Q16. What did or do you plan to do with the information you received from the GARD Information Center? (select all

that apply)

1. Review it, but do not do anything else with it. //exclusive response; once selected other options are no longer

selectable and any previous markings in these boxes are removed//

2. Make contact with individuals and/or organizations identified.

3. Share the information with my health care provider(s).

4. Share the information with my patient(s).

5. Share the information with family members or friends.

6. Other (specify) [Q16_other] ________________// FORCE SPECIFY RESPONSE//

Q16. ¿Cuáles son sus planes con la información que recibió del Centro de Información GARD?

(seleccione todas las opciones que se apliquen)

1. Revisarla, pero no hacer nada más.

2. Entrar en contacto con los individuos y/u organizaciones que fueron identificados.

3. Compartir la información con mi (s) proveedor (es) de servicios de salud.

4. Compartir la información con mi (s) paciente (s).

5. Compartir la información con parientes o amigos.

6. Otras (especifique)

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q17. Overall, how satisfied were you with the GARD Information Center? (select one)

1. Not at all satisfied

2. Somewhat dissatisfied

3. Neither satisfied nor dissatisfied

4. Somewhat satisfied

5. Completely satisfied

Q17. En general, ¿Cuál es su nivel de satisfacción con el Centro de Información GARD?

(seleccione una opción)

1. Completamente insatisfecho

2. Más o menos insatisfecho

3. Indiferente

4. Más o menos satisfecho

5. Completamente satisfecho

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q18. How likely are you to use the GARD Information Center again if you have a question related to a genetic or rare

disease? (select one)

1. Not at all

2. Unlikely

3. Likely

4. Definitely

Q18. ¿Cuál es la probabilidad de que usted utilice en el futuro el Centro de Información GARD si

usted tiene una pregunta relacionada con una enfermedad genética o rara? (seleccione una opción)

1. Ninguna

2. Poco probablemente

3. Muy probablemente

4. Definitivamente

//A

sk all//

//max characters 5000//

Q19. Please provide any suggestions that you feel would improve the information you received from the GARD

Information Center.

Q19. Por favor proporcione cualquier sugestión que usted crea que nos puede ayudar a mejorar la información que usted

recibió de nuestro Centro de Información GARD.

//As

k all//

//max characters 5000//

Q20. Please provide any additional comments about the GARD Information Center.

Q20. Por favor déjanos saber cualquier otro comentario adicional sobre el Centro de Información GARD.

----

---------//PAGE BREAK//

We’

d like to know a little more about individuals like you who use the GARD Information Center so that we can

better meet your needs.

Nos gustaría saber un poco más acerca de las personas como usted, que utilizan el Centro de Información GARD

para que podamos satisfacer mejor sus necesidades.

//As

k all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q21.Do you currently reside in the United States? (select one)

1. Yes (specify state) [Q21_state] ________________// DO NOT FORCE SPECIFY RESPONSE//

2. No (specify country where you reside) [Q21_country] ________________// DO NOT

FORCE SPECIFY RESPONSE//

Q21. ¿Vive usted actualmente en los Estados Unidos? (seleccione una opción)

1. Si (Especifique el estado donde vive)

2. No (Especifique el país de su residencia)

//Ask all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q22. Which of the following best describes the area you live in? (select one)

1. Urban

2. Suburban

3. Rural

Q22. ¿Cuál de las siguientes opciones describe mejor la zona donde usted vive? (seleccione una

opciόn)

1. Urbana

2. Suburbana

3. Rural

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q23. What language do you prefer to speak? (select one)

1. English

2. Spanish

3. Other (specify) [Q23_other] ________________// FORCE SPECIFY RESPONSE//

Q23. ¿Qué idioma prefiere hablar? (seleccione una opción)

1. Inglés

2. Español

3. Otro (especifique)

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q24. What language do you prefer for written materials? (select one)

1. English

2. Spanish

3. Other (specify) [Q24_other] ________________// FORCE SPECIFY RESPONSE//

Q24. ¿Qué idioma prefiere para sus respuestas por escrito? (seleccione una opción)

1. Inglés

2. Español

3. Otro (especifique)

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q25. What is your gender? (select one)

1. Female

2. Male

Q25. ¿Cuál es su género? (seleccione una opción)

1. Femenino

2. Masculino

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q26. What is your age? (select one)

1. 20 or under

2. 21-30

3. 31-40

4. 41-50

5. 51-60

6. 61-70

7. 71 or over

Q26. ¿Cuál es su edad? (seleccione una opción)

1. 20 años o menos

2. 21-30 años

3. 31-40 años

4. 41-50 años

5. 51-60 años

6. 61-70 años

7. 71 años o más

//Ask all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q27. What is the highest level of education you have completed? (select one)

1. Less than high school

2. Some high school

3. High school diploma or equivalent

4. Some college, no degree

5. Postsecondary non-degree award

6. Associate's degree

7. Bachelor’s degree

8. Master’s Degree

9. Doctoral or professional degree

Q27. ¿Cuál es el nivel más alto de educación que ha completado? (seleccione una opción)

1. Educación primaria

2. Educación secundaria incompleta

3. Educación secundaria o equivalente

4. Alguna educación universitaria, sin diploma

5. Educación post-secundaria, sin diploma.

6. Diplomado

7. Licenciado

8. Maestría

9. Doctorado o titulo profesional

//A

sk all//

//single response//

//*88 = NO ANSWER (HIDDEN)//

Q28. What is your ethnicity? (select one)

1. Hispanic or Latino

2. Not Hispanic or Latino

Q28. ¿Cuál es su origen étnico? (seleccione una opción)

1. Hispano o Latino

2. No Hispano o Latino

//As

k all//

//multiple response//

//*88 = NO ANSWER (HIDDEN)//

Q29. What is your race? (select all that apply)

1. American Indian or Alaska Native

2. Asian

3. Black or African American

4. Native Hawaiian or Other Pacific Islander

5. White

Q29. ¿Cuál es su raza? (seleccione todas las opciones que se apliquen)

1. Indígena Americano o Nativo de Alaska

2. Asiática

3. Negra o Africo-Americana

4. Nativo de Hawái o de otras Islas del Pacifico

5. Blanca

----

---------//PAGE BREAK//

hank you for your time today. If you have any questions about this survey, please contact the Help Desk at 1-888-

205-2311 or the GARD Information Center INFO@NIH.GOV

//hyperlink//.

Gracias por su tiempo. Si usted tiene preguntas sobre esta encuesta, por favor póngase en contacto con el Servicio

de Asistencia y llamé el número: 1-888-205-2311 o a envié un correo electrónico al Centro de Informaciόn de

GARD: INFO@NIH.GOV

Appendix B

Report of Survey Pretest Results

Customer Satisfaction Survey of GARD

Report of the Results from the Survey Pretest

September 2013

Data Analysis and Report by Lisa S. Lanier, Ph.D., M.P.H.

Evaluation Consultant

Pretest Purpose

The purpose of the pretest was to:



Identify any survey questions that respondents did not understand or in some way found difficult to answer.

Identify any survey formatting or response flow difficulties that respondents may have experienced.

Pretest Methodology

•

Phone interviewers and GARD Information Specialists were trained.

Pretest was conducted July-August 2013 with the goal of obtaining no more than nine complete surveys

(approximately half English and the other half Spanish).

GARD inquirers who received an email response to their questions also received a link to the pretest survey.

Pretest survey respondents were instructed to use the comments fields at the end of the survey to not only

provide comments about their experience with GARD but also provide comments about how to improve the

survey.

Due to a low response rate, it was decided to seek pretest participants from individuals who contacted

GARD by phone.

Due to the difficulty of obtaining survey completes, the pretest was stopped after a total of five completes

(four English and one Spanish).

Recommendation

Based on a review of the pretest data, I recommend that no changes be made to the survey questions or data

collection system. The survey and proposed data collection strategy is ready for submission to OMB for generic

clearance.

Pretest Results

These results reflect the survey responses of five individuals who contacted GARD for information. Four of the

individuals spoke English and one spoke Spanish. Two respondents reviewed Q&As on the website and then

submitted the online email form. One respondent contacted GARD by regular email and received his/her

response by email. Two respondents called GARD and spoke to an Information Specialist.

NOTE: For each survey question, the value or option and the associated count (i.e., number of respondents that

chose that value or option) is provided.

Question 1. How did you first find out about the GARD Information Center?(select one)



Search engine (e.g. Google, Yahoo, etc.) – 2

Office of Rare Diseases Research (ORDR) website – 2

Health care provider – 1

Question 2. How many times did you use the GARD Information Center to obtain information in 2012?(enter

value)



Value of 0 – 5

Question 3. How many times have you used the GARD Information Center to obtain information in 2013,

including your most recent attempt?(enter value)



Value of 1 – 3

Value of 4 – 2

The following questions refer to your most recent interaction with the GARD Information Center (e.g., Q&A on

website, contact with GARD Information Specialist through email, telephone, fax, etc.).

Question 4. Which of the following best describes you?(select one)



Patient – 3

Family or friend of patient – 1

Other – (specified FAMILIAR DEL ENFERMO, which is family of patient) – 1

Question 5. What type of information were you looking for? (select all that apply)



Alternative therapies – 1

Diagnosis for symptoms – 1

Genetic services referral – 2

Clinical/genetic testing – 2

Research study/clinical trial – 3

Specific disease or condition – 1

Treatment/medical care – 3

Question 6. Did you try to obtain information from another source before using the GARD Information Center?

 Yes – 3

These 3 respondents answered Q7, 8, and 9.

Question 7. Where did you do your research/get your information before contacting GARD? (select all that

apply)



Website – 1 (did not specify the website)

Organization – 1 (specified - FADEPOF, which is the Argentina Federation of Uncommon Diseases)

Physician, nurse, or other health care provider/professional – 1

Other – 1 (did not specify)

Question 8. Overall, how satisfied were you with the information you received from those sources?



Somewhat satisfied – 1

Completely satisfied – 2

Question 9. Why did you contact GARD after researching your question/getting information from other

sources? (select all that apply)



For more information overall – 1

For more in-depth information about a specific topic (e.g., treatment options) – 2

To talk to an information specialist – 1

Question 10. Did you review a Q&A on the GARD Information Center website to get information you were

looking for?



Yes – 2

No – 3

NOTE: The two respondents who answered Yes to Q10 answered Q12 and then Q13.

Question 12. Please tell us what you think about the Q&A you reviewed on the GARD website. (select one

answer for each statement)



Answered my question – 1 (not at all); 1 (very much)

Gave me new information – 1 (not at all); 1 (very much)

Gave me useful information – 1 (not at all); 1 (very much)

Gave me information I could trust – 1 (not at all); 1 (very much)

Met my needs – 1 (not at all); 1 (somewhat)

Was right for my personal situation – 1 (not at all); 1 (somewhat)

Was right for my reading level – 1 (not at all); 1 (completely)

Was easy to understand – 1 (not at all); 1 (completely)

Had the right amount of information – 1 (not at all); 1 (completely)

Was easy to go through – 1 (completely); 1 (did not answer)

The three respondents who answered No to Q10 (i.e., did you review a Q&A) answered Q11.

Question 11. How did you get the information you were looking for from GARD? (select one)



I received a response by email after submitting a general email to GARD – 1 (went to Q14)

I spoke to a GARD information specialist – 2 (went to Q15)

Question 13. Did you contact the GARD Information Center in addition to reviewing a Q&A?

 The two respondents who reviewed Q&As then submitted the online email form located on the GARD

Information Center website.

NOTE: Q14 was answered by the three respondents who received a response by email whether they submitted

their questions by regular email or by the online email form.

Question 14. Please tell us what you think about the response you received from a GARD information

specialist. (select one answer for each statement)



Answered my question – 1 (not at all); 1 (somewhat); 1 (very much)

Gave me new information – 1 (not at all); 1 (very much); 1 (completely)

Gave me useful information – 1 (not at all); 1 (very much); 1 (completely)

Gave me information I could trust – 1 (not at all); 2 (very much)

Met my needs – 1 (not at all); 1 (somewhat); 1 (very much)

Was right for my personal situation – 1 (not at all); 1 (somewhat); 1 (completely)

Was right for my reading level – 1 (not at all); 2 (completely)

Was easy to understand – 1 (not at all); 1 (very much); 1 (did not answer)

Had the right amount of information – 1 (not at all); 1 (very much); 1 (completely)

Was easy to go through – 1 (not at all); 2 (completely)

Came within a reasonable time – 1 (not at all); 1 (somewhat); 1 (completely)

Question 15. Please tell us what you think about the information you received over the phone from a GARD

information specialist. (select one answer for each statement)



Answered my question – 2 (completely)

Gave new information – 2 (completely)

Gave useful information – 1 (completely); 1 (did not answer)

Gave information I could trust – 2 (completely)

Gave information that met my needs – 2 (completely)

Gave information right for my personal situation – 2 (completely)

Gave information that was easy to understand – 2 (completely)

Gave the right amount of information – 2 (completely)

Was sensitive to my needs – 2 (completely)

Question 16. What did or do you plan to do with the information you received from GARD? (select all that

apply)



Make contact with individuals and/or organizations identified – 2

Share the information with my health care provider(s) – 2

Share the information with family members or friends – 1

Other – 2 (specified - evaluar que modos de acciones seguir, which is evaluate modes of action to continue);

(specified - person hung up)

Question 17. Overall, how satisfied were you with the GARD Information Center?

 Not at all satisfied – 1



Somewhat satisfied – 1

Completely satisfied – 3

Question 18. How likely are you to use the GARD Information Center again if you had a question related to a

genetic or rare disease?



Not at all – 1

Unlikely – 1

Likely – 1

Definitely – 2

Question 19. Please provide any suggestions that you feel would improve the information you received from the

GARD Information Center.



“It would be nice to have someone polite and professional answering the phone.”

“The online response was excellent information and very appreciated. I will definitely followup with the Dr.

you recommended. However, when I called on the phone, an impolite person provided little information and

hung up on me … or we were disconnected.”

“nade que sugerir, totalmente agradecido” … nothing to suggest, totally grateful

Question 20. Please provide any additional comments about the GARD Information Center.



Translation from comment written in Spanish … “exceptional willingness to provide information to people

who really need it”

“I appreciated the online response. I did not appreciate the person I called on the phone.”

“The women I spoke with, Sara, was extremely helpful. She told me that she needed to do some research

and called me back within an hour. She found a contact person for me and gave me lots of information. She

made me feel like she cared about my son’s situation and was going to help me to the best of her abilities. I

felt really good about our future with the NIH GARD.”

Questions 21-28 were demographic questions. What follows is the profile of the five pretest survey respondents.



4 reside in the U.S. (MO and Oregon) and 1 resides in Argentina

1 lives in an urban area and 4 in a rural area

4 prefer to speak and have written materials in English and 1 prefers to speak and have written materials in

Spanish

4 respondents are female and 1 is male

1 is between the ages of 21-30; 3 are between the ages of 51-60; 1 did not answer

1 has some college, no degree; 1 has an associate’s degree; 2 have doctoral or professional degree; and 1 did

not answer

1 is Hispanic or Latino and 3 are not Hispanic or Latino

4 indicated their race as white and 1 did not answer

Appendix C

Email Response with Survey Information

Dear Ms.,

You recently sent an email to the Genetic and Rare Diseases (GARD) Information Center about triple X

syndrome. You mentioned that your daughter has this condition and would like to have a baby. You asked if it

would be difficult for her to get pregnant and if the child would also have triple X syndrome.

We have answered a very similar question about triple X syndrome on the following page:

http://rarediseases.info.nih.gov/gard/5672/47-xxx-syndrome/case/29038/case-questions

We hope that the information on this page is helpful. You can use the left menu for more information and links

to resources about this condition. We also suggest that you discuss this information with a medical professional.

GARD is currently conducting a Customer Satisfaction survey, and we would love to hear from you. You will

be receiving a separate email later today containing the survey link and information about the survey. We hope

you will choose to participate.

Sincerely,

Michelle Greenwood

Information Specialist

The Genetic and Rare Diseases (GARD) Information Center is funded by two parts of the National Institutes of

Health (NIH): the National Center for Advancing Translational Sciences Office of Rare Diseases Research

(NCATS-ORDR) and the National Human Genome Research Institute (NHGRI). The GARD Information

Center provides the public with access to current, reliable, and easy to understand information about rare or

genetic diseases in English or Spanish. Information Specialists are available by phone Monday through Friday,

12:00 p.m. to 6:00 p.m. Eastern time (excluding Federal holidays).

PO Box 8126

Gaithersburg, MD 20898-8126

Toll-free: 1-888-205-2311

Telephone: 301-251-4925

TTY: 1-888-205-3223

Fax: 301-251-4911

E-mail: [email protected]ov

Web site: http://rarediseases.info.nih.gov/GARD

Important Disclaimer:

The materials provided are for informational or educational purposes only and are not intended as a substitute

for professional medical care, advice, diagnosis, or treatment. This material does not represent an endorsement

of any specific tests or products by the National Center for Advancing Translational Sciences Office of Rare

Diseases Research (NCATS-ORDR) or the National Human Genome Research Institute (NHGRI). We cannot

guarantee the accuracy, completeness, timeliness, or usefulness of the opinions, advice, services, or other

information. Moreover, we strongly recommend that you seek the advice of your health care provider with any

questions regarding your medical care.

45352

Dear ,

We would like to invite you to participate in the GARD Customer Satisfaction Survey.

Public reporting burden for this collection of information is estimated to average 10 minutes per response,

including the time for reviewing instructions, searching existing data sources, gathering and maintaining the

data needed, and completing and reviewing the collection of information. An agency may not conduct or

sponsor, and a person is not required to respond to, a collection of information unless it displays a currently

valid OMB control number. Send comments regarding this burden estimate or any other aspect of this

collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch,

6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0648). Do not return the

completed form to this address.

The survey asks questions regarding your satisfaction with the services you received from the GARD

Information Center, as well as how you use the information GARD provides. Additionally, at the end of the

survey there are several comment boxes. Please feel free to leave any additional feedback regarding the survey

itself in these boxes. Information from this survey will be used to help improve our responses to the questions

we receive.

The survey is available in English or Spanish. The survey is completely voluntary and anonymous.

Please use the following link to access the survey:

https://spss.icfsurveys.com/mrIWeb/mrIWeb.dll?I.Project=GARD

Thank you for your participation!

Sincerely,

Pamela McInnes, DDS, MSc.(Dent.)

Deputy Director, NCATS

Acting Director, Office of Rare Diseases Research

National Center for Advancing Translational Sciences, NIH

6701 Democracy Blvd., Rm. 904

Bethesda, MD 20892-4874

Phone: (301) 435-0877 Fax: 301-482-3654

[email protected]

Appendix D

Results of Analysis of Spanish Responses

Results – Analysis of Spanish Responses

During the data collection period, 930 surveys were completed of which 98.0% (n=911) were in English and

2.0% (n=19) in Spanish. Below are the results of an analysis of the 19 responses that were in Spanish. Note:

The percent reported is the valid percent (i.e., only the percentage of respondents who answered the question).

Survey Objective 1. Identify who is being served by the GARD Information Center

Responses to survey questions 4 and 21-29 were designed to provide a profile of the survey respondents. The

profile of the 19 respondents who completed the survey in Spanish was as follows:

Respondents Role (relationship to the disease) when Contacted GARD

Role

Percent

Patient

21.1%

Family or friend of patient

21.1%

Student

21.1%

Physician

15.8%

Other (i.e., consultant, regional org for rare diseases)

10.5%

Other health care provider or allied health professional

5.3%

Residence

•

73.7% of respondents resided outside of the United States (U.S.) in the following countries: Spain (n=5),

Mexico (n=2), Argentina (n=2), Brazil, Columbia, Ecuador, and Switzerland.

21.1% of respondents resided in the U.S. or a U.S. territory (Florida, New Jersey, Puerto Rico).

73.7% of respondents indicated they lived in an urban area, 15.8% lived in a suburban area, and 5.3% in a

rural area.

Language

• Of the 19 Spanish survey respondents, 84.2% (n=16) preferred to speak and read in Spanish, 5.3% (n=1)

preferred to speak and read in English, and 5.3% (n=1) preferred to speak and read in Portuguese.

Gender, Age, and Education

•

63.2% of respondents were female and 26.3% were male.

The tables below show the percent of respondents by age and by highest level of education completed.

•

Age

Percent

21-30

26.3%

31-40

31.6%

41-50

31.6%

Education

Percent

HS diploma/equiv

5.3%

Some college, no degree

15.8%

Associate’s degree

31.6%

Bachelor’s degree

26.3%

Master’s degree

10.5%

Doctoral/Professional

5.3%

Ethnicity and Race

89.5% (n=17) of respondents indicated that they were Hispanic or Latino and 5.3% (n=1) indicated that they

were not Hispanic or Latino.

78.9% (n=15) of respondents indicated their race as White and 5.3% (n=1) indicated Black or African

American.

Who is being served by the GARD Information Center?

A snapshot of the Spanish survey respondents:

•

Patient, family member/friend of a patient, or student

Female

Age 31-50

Associate’s degree

Hispanic or Latino

White

Prefers to speak and read in Spanish

Resides in an urban area of a Spanish-speaking country

Survey Objective 2. Identify how individuals learn about the GARD Information Center

Survey questions 1, 2 and 3 were designed to provide information about how respondents learned about the

Information Center.

Visits to GARD

•

Approximately 59% of Spanish survey respondents did not visit the Information Center in 2013. Of the

survey respondents who visited the Center in 2013, approximately 18% visited one time. Four respondents

indicated that they visited the Center more than once in 2013.

Approximately 39% of Spanish survey respondents visited the Center one time in 2014 (includes visit in

which respondent was asked to complete the survey). Approximately 17% of survey respondents indicated

that they visited the Center two times in 2014 (between January 1, 2014 and April 25, 2014). Seven

respondents indicated that they visited the Center more than twice in 2014.

Find Out About the GARD Information Center

Survey respondents identified how they first found out about the Information Center. The responses were:

Resource

Percent

Search engine (e.g., Google, Yahoo, etc.)

78.9%

Other website (i.e., Genetics Home Reference)

5.3%

NCATS/ORDR website

5.3%

Other (i.e., contact with ORDR Director)

5.3%

How do individuals find out about the GARD Information Center and how often do they visit GARD?

A snapshot of the Spanish survey respondents:

•

By searching via search engine (Google, Yahoo).

Did not seek information from the Center in 2013.

Did seek information from the Center one time between January and April 2014.

Survey Objective 3, 4, and 6. Identify the type of information being sought and the reasons why individuals are

seeking information from the GARD Information Center, and where else customers may have sought

information

Survey questions 5-9 address survey objectives 3, 4, and 6.

The type of information that the Spanish survey respondents were seeking from the Center was as follows:

•

Type of Information

Percent

Diagnosis for symptoms

52.6%

Clinical/genetic testing

36.8%

Specific disease or condition

36.8%

Research study/clinical trial

31.6%

Alternative therapies

26.3%

Genetic services

26.3%

Treatment/medical care

21.1%

Other (i.e., all available info, number of rare diseases)

15.8%

Newborn screening

10.5%

89.5% of the Spanish survey respondents indicated that they tried to or did obtain information from another

source before seeking information from the GARD Information Center. Where the survey respondents

received their information is identified below.

Location of Information

Percent

Website

77.8%

Physician, nurse, other health care provider

33.3%

Organization

16.7%

Family member/friend/colleague

16.7%

Genetic counselor

5.6%

Websites identified included: CDC, OMIM.org, Orphanet, Mayo Clinic, PubMed, Google, Wikipedia, FDA,

and CFC International

Organizations identified included: Alliance of Rare Diseases and Registries, NAF, NSGC, and NORD

Survey respondents who obtained information from another source before contacting the Center were asked

how satisfied they were with the information they received from the other sources. The responses were as

follows:

Not at all satisfied – 22.2%

Somewhat satisfied – 50.0%

Completely satisfied – 22.2%

These same survey respondents (i.e., those who obtained information elsewhere before contacting the

Center) were asked why they sought information from the Center after obtaining information elsewhere.

The responses were as follows:

For more information – 66.7%

For more in-depth information about a specific topic – 50.0%

To talk to an information specialist – 33.3%

Other (i.e., receive current info, to complement info you do not provide) – 11.1%

What types of information are individuals seeking from the GARD Information Center and what other sources

were used to obtain the information?

A snapshot of the Spanish survey respondents:

Top four types of information being sought: diagnosis for symptoms, clinical/genetic testing, specific

disease/condition, research study/clinical trial.

Majority did obtain information from another source before contacting the Center.

Majority sought information from a website other than the Center’s web pages.

•

Half of the survey respondents indicated being somewhat satisfied with the information they received from

other sources.

The top reason for contacting the Center after seeking information elsewhere was to obtain more

information overall.

Survey Objective 5. Assess customer’s satisfaction with the GARD Information Center’s services (i.e.,

responses provided by information specialists and information available on the Center’s web pages)

Survey questions 10-20 address customer satisfaction.

Satisfaction with Information from Disease-Specific Information Pages (Q&A)

47.4% of the Spanish survey respondents indicated that they reviewed a Q&A on the GARD website to get

information they were looking for. Below is what those respondents thought of the information they were

provided in a Q&A.

•

The information…

Not at all

Somewhat

Very Much

Completely

answered my question(s)

22.2%

55.6%

22.2%

gave me new information

22.2%

44.4%

33.3%

gave me useful information

11.1%

66.7%

22.2%

gave me information I could trust

11.1%

44.4%

met my needs

22.2%

66.7%

11.1%

was right for my personal situation

44.4%

33.3%

22.2%

was easy to read

11.1%

44.4%

was easy to understand

11.1%

55.6%

33.3%

had the right amount of information

44.4%

22.2%

33.3%

was easy to go through

44.4%

33.3%

22.2%

Approximately 67% of the Spanish survey respondents who reviewed a Q&A indicated that they did not

contact the Center (via online contact form, email, phone, or letter) after reviewing a Q&A.

Satisfaction with Information Received by Email or Letter

Of the survey respondents who did not receive information from the Center by reviewing a Q&A, 30.0%

received information by email after submitting the online contact form. Below is what those respondents

thought of the information they were provided.

Satisfaction with Information from an Information Specialist by Phone

Note: None of the Spanish survey respondents indicated that they spoke to an information specialist.

The information …

Not at all

Somewhat

Very Much

Completely

answered my question(s)

33.3%

50.0%

gave me new information

16.7%

50.0%

16.7%

gave me useful information

50.0%

33.3%

gave information I could trust

50.0%

33.3%

met my needs

16.7%

33.3%

was right for my personal situation

50.0%

33.3%

was easy to read

50.0%

33.3%

was easy to understand

50.0%

33.3%

had the right amount of information

50.0%

33.3%

was easy to go through

16.7%

50.0%

16.7%

came within a reasonable time

50.0%

33.3%

What Will Respondents Do With the Information?

All survey participants were asked what they did or plan to do with the information they received from the

Center. The responses from the Spanish survey respondents were:

Responses identified under ‘other’ included: final school project, write a lab report, write a project, and use you

for help on my school information.

Overall Satisfaction with the GARD Information Center

Survey respondents were asked to indicate how satisfied they were with the GARD Information Center.

Responses were:

Action

Percent

Review it, but do not do anything else with it.

0.0%

Make contact with individuals/organizations identified.

42.1%

Share information with health care providers.

10.5%

Share information with patients.

10.5%

Share information with family members or friends.

10.5%

Other

36.8%

Satisfaction

Percent

Not at all satisfied

10.5%

Somewhat dissatisfied

5.3%

Neither satisfied nor dissatisfied

10.5%

Somewhat satisfied

42.1%

Completely satisfied

26.3%

Use of the GARD Information Center Again

Approximately 53% of the Spanish survey respondents indicated that they were likely to use the Center again,

approximately 32% indicated they would definitely use it again, and approximately 10% indicated they were

unlikely to use it again.

Assessment of satisfaction with GARD Information Center services

A snapshot of the Spanish survey respondents:

•

The Spanish survey respondents received information from GARD by either reviewing a Q&A or by

submitting the online contact form and receiving information by email.

Of the survey respondents who reviewed a Q&A, the majority did not seek further information from the

Center.

Survey respondents who reviewed a Q&A expressed a high degree of satisfaction (as indicated by

responding ‘very much’ or ‘completely’ to the questions) with the information they received. The

respondents indicated ‘somewhat’ to the following questions: was right for my personal situation, had the

right amount of information, and was easy to go through.

Survey respondents who received information by email (following submission of the online contact form)

expressed a high degree of satisfaction (as indicated by responding ‘very much’ or ‘completely’ to the

questions) with the information they received.

Overall, survey respondents were somewhat satisfied (as indicated by 42.1% of respondents) with the

GARD Information Center.

Survey respondents indicated that they would most likely make contact with individuals or organizations

identified in the response.

Just over half (53%) of survey respondents would likely contact the Center again.

Suggestions for Improving Information Received from the GARD Information Center and Additional Comments

About the Information Center

Below are the comments and suggestions from the 19 Spanish survey respondents:

•

“From GEISER, we think that we could develop a collaborative project to incorporate specific information

useful to Latin America.”

“Options to change the language: English/Spanish for all the information.”

“Gather an area of definitions and references to know what we are talking about. Have access to the entire

list and not (a list) divided by letters of the alphabet.”

“It isn’t only important to have the information in the same language of the inquirer, but you also need to

adapt the language based on the different cultures for it to be of true help.”

Limitations, Conclusion and Recommendations

The 2014 evaluation of GARD had the following limitations: use of a convenience sample, insufficient pretest

data, and no interim data. In addition, the convenience sample had an insufficient number of Spanish speakers.

These limitations contribute to the lack of ability to generalize with confidence to the larger GARD population

and unknown introduction of bias in the responses.

Although the number of Spanish responses was insufficient, the following data may be helpful to consider when

discussing changes to GARD and, in particular, how information is presented to Spanish-speaking individuals.

•

Included in the pool of Spanish respondents were students.

The Spanish respondents were younger in age (31-50) as compared to the English respondents (51-60).

The Spanish respondents were less educated (Associate’s degree) as compared to the English respondents

(Bachelor’s degree).

The majority of Spanish respondents resided in an urban area of a Spanish-speaking country as compared to

the majority of English respondents who resided in a suburban area in one of the southern or western United

States.

The Spanish-speaking respondents indicated that clinical/genetic testing information was one of four top

reasons for contacting GARD; whereas, this reason was not one of four top reasons indicated by the

English-speaking respondents (treatment/medical care was in top four).

The Spanish-speaking respondents indicated that they plan to contact the individuals/organizations

identified in the information from GARD; whereas, the English- speaking respondents indicated that sharing

the information with others (e.g., family members, health care providers) was the action they were taking

with the information.

Although it appears (based on the data) that the Spanish respondents received information from GARD by

only reviewing a Q&A or submitting the online contact form, there were seven respondents who did not

answer the question pertaining to how they received information. It is known from other data collection

opportunities (both internal and external to GARD) that Spanish-speaking individuals do contact GARD by

phone as well.

Consideration should be given to the amount of information on the GARD website that is available in

Spanish.

Consideration should be given to the nuances that exist in the cultures and traditions of the many Spanish-

speaking countries.

Appendix E

Individual Suggestions for Improving Information Received from GARD and

Additional Comments about the GARD Information Center

Note: Grammatical errors in the original comments were corrected for in this report.

•

Use less technical or difficult language/terms

“Add the information in less technical terms so regular patients, their parents, and students can understand

the information. They may not have graduated or have had classes that would enable them to understand

the technical language.”

“I am a nurse so I understand the language but not sure all people will.”

“Papers should be in basic English vocabulary if possible.”

“Use simpler terminology so the information is more easily obtained through search engines.”

“The site is informative, but somewhat hard for me to understand because I’m Swedish.”

“Information seems more for the super-educated.”

“Even though I have been a RN for 35 years, I was diagnosed with a muscle disease that I personally

wanted more information about. So, from a patient’s point of view, I wanted something explained more

clearly about the disease, what to expect in the future, how to plan my life that has been radically changes,

etc.”

“Make sure deaf can read low level English and picture. YouTube for American sign language would be

helpful.”

“It isn’t only important to have the information in the same language of the inquirer, but you also need to

adapt the language based on the different cultures for it to be of true help.”

Improve the accuracy of the information

“The pericardium lines the heart. The peritoneum lines the abdomen. In the article Pentalogy of Cantrell,

your information called the pericardium the lining of the abdomen.”

“I wrote to GARD not for information but to inform you that I felt the description for Orthostatic Tremors

needs to be updated because it is inaccurate.”

“Not convinced the information on treatment for Cold Agglutinin disease (CAD) is correct—use of

Prednisone and removal of spleen are not recommended for this illness by main commentators/specialists.”

“The information suggesting that GARD is a resource for polymyalgia rheumatic did not match up with the

GARD response that it does not consider this rare. Conflicting information. It only directed me to juvenile

polymyalgia rheumatic.”

“Care needs to be taken to ensure information is correct and reflects best practice for specific diseases.”

Update the information

“Please update the Spontaneous Coronary Artery Dissection section to link to www.scadalliance.org and

our facebook page, https://www.facebook.com/pages/Spontaneous-Coronary-Artery-Dissection-SCAD-

Alliance/229928783729665”

•

“On further inquiry, I found out that the info on clinical trials is passively updated, i.e., relies on CT

project. Realize the expense of NIH doing it though wonder about that with current IT. Some info was

woefully out of date.”

“Not enough current, relevant information about symptomatic pineal cyst.”

“If there are any updates on TMAU research/treatments, please publish it on the portal.”

“I was searching for information on metaplastic carcinoma of the breast and the information was very

general and there were no specific groups listed of use. The page hasn’t been updated in 3 years.”

“It would be helpful for the agents to have the most current information on study requirements and needs.”

“Have an updated list of referrals if providers could update what they do with the disease. In my case, only

one of the researchers was accepting general questions. The other one had just been running the lab. I

needed a medical opinion.”

“You should update the database every year or six months, especially in regards to new clinical trials.”

“Very limited and out of date. Sent some suggestions over 6 months ago. Nothing done.”

“I am pleased that someone read my note and did further research and updated the description but I still

don’t feel that it’s complete.”

“Some information needs to be updated more frequently.”

Provide information about a particular disease or topic (i.e., information currently not provided or

can’t be found on the website)

“Can’t find info on my diagnosis of Rhabdomyomatous Mesenchymal Harmartoma of the tongue.”

“Baffled how you can call yourself a place for resources for people with rare diseases. The condition I have

isn’t even listed anywhere on the site (Jackhammer Esophagus). It is definitely rare.”

“Not much information here for many diseases. Appears to be very patient/clinician oriented, but others are

directed here as well from FDA and ORDR sites. Should broaden viewpoint.”

“GARD should reply to the queries asked related to genetic disorders in the prenatal period.”

“It would be nice if some of the rare, genetic disorders would add how it may affect people as they mature –

what delays may occur.”

“Please include the very rare diseases.”

Provide more information about a particular disease or topic (i.e., not enough information currently

provided on the website)

“More information about Porphyria disease. It’s so rare that maybe there wasn’t enough information for

GARD.”

“More information regarding adults with mito diseases”

“Would like to see more info about CPEO Plus.”

“You have little to no information on lipedema genetic research.”

“There really isn’t much information anywhere about IBM (Inclusion Body Myositis).”

“I did a search on Lynch Syndrome and found the results to be pretty generic. I am looking for more in

depth discussion of other disease associations in patients that have Lynch Syndrome.”

“I was searching for more information on Mucoepidermoid carcinoma but it was just a brief statement.”

“Looking for information on Logopenic Progressive Aphasia.”

“There was little information about systemic candidiasis.”

“He wanted more information. What is the treatment plan for HTLV, life expectancy, new treatments. What

are the early symptoms and what is the window period?”

“Cramp Fasciculation Syndrome article was very short. Not useful.”

“Organized list of institutions to treat Cutaneous T-Cell Lymphoma and other rare disease, even outside of

America.”

“I suffer from Wolff-Parkinson-White Syndrome. Not sure if rare, but I live in Africa where health care is

very mediocre and I can’t get proper advice and treatment. I thought your organization would help even if

can’t help with treatment at least give professional advice.”

• Provide or link to other types of information (or additional information)

“When making quotes regarding acceptable levels for lab studies, it would be helpful to post what the

normal expected levels are.”

“Difficult if a person has more than one rare condition. Would be good to create a reference for ‘if you

have x and y other symptoms/complications to look for might be …’

“List medical treatments.”

“Please provide easy to find epidemiological information on rare diseases when possible.”

“More data on side effect for treatment and medications mentioned on this site or links to sites that could

provide info.”

“Did not get information on morbidity and mortality.”

“It would be nice to have a list of physicians on the websites directly. Something that said ‘there is no

specific provider list for this disease, but if you call we can tell you how we find providers.’ Guide on how to

find a provider for people who don’t want to call.”

“I will look at the site to see if you have supporting documentation for use by librarians serving consumer

health information.”

“Better link system for patients and physicians with the main NIH sites might help a bit.”

“Would have liked an actual specialist referral. List of all rare diseases and the physicians so we can call

them directly.”

“More resources to refer to.”

“Better linkage to other centers of excellence around the world, i.e., be less U.S. centric in terms of

information management.”

“It would be great if there were information for teachers of children with rare disorders to help understand

their needs in the classroom.”

“Photographs to illustrate specific features of a particular syndrome. Perhaps you have them, but not for

Jeune Syndrome.”

“More pictures if there are characteristic physical manifestations of the disease.”

“Add examples or some cases about the genetic or rare disease.”

“It would be nice if patients could find out where they can go for testing for a rare disease.”

“Provide research centers that the person could be a part of for what they were searching.”

“Health care referrals to caring people with knowledge of effective treatments.”

“If possible, would request a little more information such as leading research being done in the field. To

find out there isn’t any hope for your condition is really hard to take.”

“More in-depth information needed for specific subjects, or at least links to other trusted sites.”

“More on home cures.”

“GARD should include whether specific symptoms are related to the condition I’m inquiring about or

whether there was another cause.”

“Need more information on how to control pain and depression.”

“I would like to know about lifestyle/dietary changes that I can make.”

“Would like more info to understand the statistics of these rare cancers and diseases.”

“Links to support groups if known.”

“Need a survey of people suffering from the same disease and their prognosis.”

“It would be helpful if a patient had a check list to take to a doctor. The doctors don’t ask the right

questions. I know that they are missing the overall picture.”

“Identify strategies for promoting one’s MD’s interest in learning more about a particular rare disorder

and for educating one’s personal MDs about a particular rare disorder.”

•

Modify structure/format of response

“Clearer guidance to more comprehensive information. For many users MedLine may be daunting. Some

categorization of the articles may be helpful.”

“The Q&A was a little bland, but the specialist definitely helped. It’s great to be able to call someone.”

“I would like to see the results of the studies that I found.”

“Provide pages with information rather than asking visitors to contact you.”

“More information and links.”

Modify website capabilities/features

“I would like to find a button that could keep me updated on new information about the rare disease I’m

searching.”

“Have a tab for physical effects and a tab for prognosis if known and possibly a link to case studies of the

disease.”

“It would be wonderful to be able to pull up a reference list of rare diseases according to GARD that have

to do with specific organ systems (like lungs).”

“Access to links should be highlighted once not 15 times in one screen.”

“Ability to search specifically on a symptom and find associated rare disorders.”

“Facilitate more complex searches.”

“It would be very beneficial to be able to search for a doctor to see.”

“Need an easier way to search condition other than just alphabetically.”

“For rare diseases, please provide links to as many additional resources as possible. Also, it would be

helpful to have a blog and/or forum on the site or links to actual people with the rare disease to

communicate with each other/support.”

“Options to change the language: English/Spanish for all the information.”

“Greater amount of info on home page, one small paragraph appears non useful.”

“The interface is great if it is just to clarify whether a certain disease is rare or not (and this is what I have

been using it for predominantly). However, in some cases I was looking for more information (such as

prevalence) and GARD only offered links to other websites. Since I had to check several other diseases this

would have been very time consuming.”

“Gather an area of definitions and references to know what we are talking about. Have access to the entire

list and not a list divided by letters of the alphabet.”

“You should provide a recommended citation for the database for researchers who wish to refer to it as a

resource.”

“General health information on one of the links was relatively useless. Some links were dead. Website

maintenance should be conducted annually.”

“The website is not visually appealing. It is adequate, but could be spruced up.”

“Very slow and ineffective. Do not support 90% of rare diseases. Only providing support to most common

ones.”

•

“More conditions/diseases, a section defining medical terms, and pictures/photos/images.”

“Have a look at other websites that provide information to see the redundancies that enable more than one

pathway to the data/information you have available. Greatly simplify the access. Your site is tight and

restricted. It is discouraging. Little return for effort extended. No information on some pages. 30 minute

waiting time …”

Improve website navigation

“The information is there, but not always clear where it is.”

“The navigation to find needed information is somewhat cumbersome. A simple search function would be

nice.”

“Not very easy to search or find information about related conditions.”

“I have not yet sent an email to ask about listings of rare diseases, but I’m hopeful that browsing such a list

will become easier in the future.”

“I didn’t realize I could email my questions, but I will now.”

“I got to the page by web search. If I hadn’t taken this survey I wouldn’t have known there was a possibility

of receiving emailed answers – the disease info page didn’t say.”

“How do I get to the GARD Information Center?”

“I never reached the GARD Information Center.”

“Nothing on this site says “GARD” that I can see. I chose the site because I know I can trust the

information I received from the NIH.”

“I don’t know what GARD Information Center is. I just found information about a disease at ORDR site.”

“I somehow found this website yet was unable to find the actual Q&A GARD site. I would be interested in

locating this site or speaking with a professional.”

“I had trouble initially finding the answer to my question in the Q&A, but once I was given a link to the

page the information was easy to find.”

“Instead of having links have different sections of the website, so the information is all there without having

to load another page.”

“When clicking on numerous links imbedded everywhere, it’s hard to keep track of where you’ve already

been as opposed to finding something new entirely.”

•

Improve response time

“Please respond more quickly.”

“It did take nearly three weeks to receive a response but I’m sure you face a very big task each day in

providing information. The site seems very patient oriented as was the reply. This is very beneficial for the

lay person. My need was more immediate since I was filing an application for an Orphan Drug Designation

….”

“The info was great and really appreciated but if you can reply sooner would be a bonus.”

Conduct outreach/advertise

“Have a commercial so people will know about the center.”

“More public outreach needed.”

“Let other agencies/organizations know that GARD exists.”

“Advertise. I did not know about GARD.”

“I consider myself to be a reasonably informed senior citizen, but was unaware of this Information Center.

Perhaps you can find ways to mainstream your services.”

“I didn’t know this site existed until I saw a link on another website.”

“I really believe the site needs more public exposure.”

Improve communication (written and oral) with GARD customers

“I needed to know about severe back pain. That was not addressed. If you do not know, tell me that.”

“More information on the disease and if no additional information then let the patient know.”

“Rare diseases and difficult diagnosis are frightening to people. Stating ‘not enough information’ doesn’t

help with that.”

“Please answer the question or tell me you are unable to.”

“If you have no details regarding a question, please state that and do not keep referring a person back to

general information.”

“The first thing the information specialist said was ‘what is your disease’. I didn’t know how to answer. I

have a deformity. Maybe I’m just sensitive. Wouldn’t it be better to say ‘how may we help you’.

“Although I am from a Spanish-speaking country, I would have preferred the answer in English. There was

some mistranslation in the answer you gave in Spanish.”

“GARD said the information I had sent was insufficient to make any suggestion. It would have been helpful

if some indication was given as to what would have been more helpful.”

“State more clearly what type of service GARD is able to provide. That way I wouldn’t have wasted the

information specialist’s time with my medical question.”

“I was expecting a more complete information center. I was not understanding it was just a quick intro and

direction to other sources.”

“I misunderstood what information GARD could provide.”

•

Other suggestions/comments

“Change acronym.”

“Information was too vague.”

“All information provided on this site can be easily found with a search engine. I can read and interpret the

medical literature myself. There was nothing new here.”

“I just needed basic information. That is what I found.”

“Site only referred me to sources I had already visited or knew about. No new information.”

“There should be a world database of all sickness symptoms and rare diseases for doctors and all society

to understand possible causes and proper treatments that are known to work. Info should be a mouse click

away nowadays to have a proper diagnosis and correct treatment.”

Appendix F

Individual Compliments about GARD

Included in the 441 suggestions/comments for improvement were many comments related to the high level of

satisfaction with the information, the Center, and the information specialists.

•

Compliments about the GARD Information Center and the information provided

“I am sure that this site will be invaluable to aid so many individuals who go undiagnosed for lack of PCP

training that is specific to their rare disease.”

“Most informative of any place contacted.”

“Really liked the audio feature.”

“You provided the best info thus far that I have found online. Please keep the audio as it made everything easier

to understand for us everyday people who are not medical pros.”

“All my needs were met.”

“Good and helpful information.”

“Very detailed website.”

“My first time on your website. Found it very informative with lots of resources. Will definitely be using your

site again.”

“I like the format.”

“Thank you for all you do for us.”

“Thank you. You provide the best access to rare disease information that helped me get to a diagnosis.”

“Thank you for giving me the information that I requested. I learned new information that I can give to my

healthcare team and family.”

“I am glad to see this site. I am always looking for helpful resources that are reliable, both for my patients and

for myself.”

“I think for people in research it is invaluable, even for other countries.”

“The GARD Information Center is an invaluable and empathetic source group helping via the Internet.”

“This is a great resource for nurses.”

“Thank you for being here for those of us who fall through the cracks of mainstream medical treatment.”

“I go to GARD for more detailed information so that I can gain a better understanding of the condition.”

“Thank you for the efforts you make to help all of us with rare diseases.”

“Great job! I love this site. Very helpful.”

“If I need clear concise information regarding my condition, you help make that job easier. You are the most up

to date information place.”

“I felt it was most helpful as I had no idea where to begin looking for information about the topic or potential

continuing educational speakers.”

“The information provided for my search was short and succinct and easy to understand – precisely what I

needed.”

“Appreciate links to so many potentially helpful organizations and availability of counselors if needed.”

“It has a good platform.”

“I appreciate the information that I get as I know it is credible.”

“Excellent information site and received reply to question very quickly.”

“It’s great knowing a parent like me who is not very educated can get information like this.”

“The most comprehensive information on the internet.”

•

Compliments about the information specialists

“Information Specialist was very knowledgeable and wonderful. Very satisfied. It was great to have someone

there who has all the information at their fingertips.”

“The information specialist was very helpful. The information I was looking for is just not out there. The

information specialist was very honest. It was great to get the newest articles. She gave names, didn’t promise

anything, but helped me understand who I needed to …”

“The specialist I spoke with was a complete pleasure to speak to and was very sensitive and knowledgeable.”

“The information specialist was very informative, very polite, and very knowledgeable. She went above and

beyond.”

“The specialist was very friendly, polite, knowledgeable, and patient.”

“The specialist was empathetic and professional.”