Joe Finnerty

QUESTION-BASED DATA COLLECTION METHODS

1. question based data-collection methods – general introduction

2. main types of question-based methods

3. postal surveys

4. response rates

5. some advantages of the face to face/personal interview

6. questionnaire construction – a rough guide

7. other issues

In these kinds of research, the researcher collects data by asking people who have experienced certain phenomena to reconstruct their experiences, describe their behaviour and express their preferences/attitudes. S/he may also ask about their knowledge of events.

Keeping in mind budget restrictions (time, money, personnel), the researcher must choose the data collection method that will elicit the most complete responses from a sample of individuals / organisations relevant to the research topic / objectives / questions.

 The responses obtained are the data on which answers to the research questions are based.

NB: distinguish between

readings

Fowler (2002), Survey Research Methods 3^rd edition ch. 5 – ‘Designing Questions to be Good Measures’;

Blaikie Designing Social Research pp. 232-5;

‘Questionnaires’ are used in (non-face to face) postal or telephone surveys and in (face-to-face) structured personal interviews. The looser set of question headings used in the qualitative interviews are often called ‘interview schedules’.

 ‘Multi-method approach’ = using more than one method e.g. combination of field research and a survey. Different kinds of questions i.e. questions that produce both quantitative and qualitative forms of data,  may be used in the same interview or quetionnaire.

1.      requires simple questions                       a postal questionnaire can be used to gather data only when the questions are straightforward enough to be understood and answered on the basis of printed instructions and definitions. In particular, the questions tend to be ‘closed’ and quantitative.

2.      no opportunity for probing        there is no opportunity to probe beyond the given answer, to clarify ambiguous answers, perhaps to appraise the non-verbal behaviour of respondents

3.      no control over who fills out the questionnaire/ no control over the interview situation                  there is no way of knowing who has filled out the questionnaire: someone other than the intended respondent may have done so.

4. low response rate                 it is often difficult to obtain a satisfactory response rate ( = the % in the sample who return completed questionnaires). Typical response rates for postal questionnaires without follow-up are 20% - 40%. The problem here is there is no way of knowing if non-respondents are different from respondents in ways that affect the findings.

SECTION 4. FACTORS AFFECTING THE RESPONSE RATE / STRATEGIES TO INCREASE RESPONSE RATES

 note:   

in the discussion below of personal interviews and postal questionnaires, it’s by and large assumed that the potential respondents are individuals to be interviewed or mailed in their own homes.

sponsorship      providing information on who has commissioned the questionnaire often motivates respondents to complete and return them. (This is usually done in the ‘accompanying / covering letter’.) This information may convince respondents  of

1.      the legitimacy of the research

2.       the usefulness of the research.

With officially sponsored research,

3.      the perceived sanctions of failure to reply may also be a factor – these may be real legal sanctions, or they may be wrongly anticipated on the part of respondents. It is of course undesirable,  and in general unethical, to suggest sanctions for failure to respond

The offer of entry into a draw overcomes this problem, but perhaps at the loss of any likely  ‘payoff’ for participation.

Mark et research is more likely to offer inducements of this kind.

Pre-paid addressed envelope

 It’s unreasonable to expect the respondent not only to spend time and effort to fill out the postal questionnaire but also to buy a stamp and write out the address on the envelope! 

Timing of mailing / interviewing

Holiday periods are bad times to send out a postal questionnaire. The time of the day that a personal interview is arranged for is important, depending on who in the household it’s intended to question.

Follow-up

the most common strategy for postal questionnaires  is to first send a reminder postcard to those who haven’t replied. Then perhaps a reminder, along with a substitute questionnaire.

However, sending a reminder may raise the question of the anonymity of the responses – as an alternative, the reminder could take the form of an announcement in the newspaper or other media.

1.       Flexibility in the questioning process

Depending on the research problem and research strategy, the interviews can vary from highly structured [in the DEDUCTIVE RESEARCH STRATEGY] to highly unstructured [ABDUCTIVE RESEARCH STRATEGY]. For focused and nondirective interviews, the interviewer can clarify questions for the interviewee, probe for specific issues raised in the course of the interview etc.

2.       Control of the interview situation

The interview gives the researcher much greater ‘control’ of the interview situation.

This isn’t as authoritarian as it sounds! -

·        Sequence of questions:   it may be desirable to ensure that questions are asked in a particular sequence. If in a survey about facilities (childcare, community centre, credit union etc.) in an area e.g. if two research questions relate to

1.     knowledge / awareness of the existence of the facility (you want to find out how aware a respondent is about the existence of facilities in their area, perhaps to gauge the success or otherwise of a publicity campaign)

2.     their use of and attitude towards the facility

-         you need to ask them question 1 before presenting them with a list of such facilities and asking them to comment on if or under what circumstances they would use the facility

·        getting the right respondent     ensuring that the respondent him/herself has answered the questions, rather than a relative – example of MS survey.

·        Timing       it may be important to fix a particular time when the data was collected: this can allows more accurate interpretation of the research results, especially when an event occurring around the time of the interview could have influenced the respondent’s answers.

3.       High response rate.

Response rates to personal interviews are usually higher than for postal questionnaires. Respondents who mightn’t bother filling out and returning a postal questionnaire will often cooperate with a request for a personal interview – there may be less effort involved on their part, e.g. if they have reading or writing difficulties.

The central component of the data collection instrument in the postal questionnaire or the structured personal interview, is the question.

The answers to the questions asked = the data on which answers to the research questions and objectives are built. The questions are the variables, which can be correlated to look for associations between variables.

 4 aspects to questions:

·        Content

·        Structure

·        Format

·        Sequence

 [1]   CONTENT OF QUESTIONS

-         Objective/factual & subjective (‘objective’ = questions that generate demographic economic data on gender, age, nos. in household, occupation etc. & questions relating to possessions and behaviour e.g. car ownership, mode of transport; ‘subjective’ = attitudes & opinions & beliefs e.g. ‘do you favour greater investment in public transport?’)

-         Behaviour & knowledge & beliefs & attitudes

 [2]  STRUCTURE OF QUESTIONS

Closed ended: respondents are offered a set of answers and asked to choose the one that most closely fits their situation/ views/etc. The set of categories will depend on the research questions.

 Open-ended: these questions aren’t followed by a set menu of choices. Rather the respondent talks or writes in their own words.

Greater flexibility & a higher response rate may be introduced by letting each individual option cover a range (of years, income etc)

Please indicate your age/ What age are you?

[   ]16 – 18                     [   ]19-24                       [   ] 25-35

[   ]36-45                       [   ]46-64                       [   ]64 or over

(spot the deliberate error!)

Some questions may be irrelevant for a particular respondent. E.g. if the respondent has no children, a question on age or level of schooling of children doesn’t apply. The questionnaire may cover a number of types of respondents, so rather than doing out as many separate questionnaires as there are respondent types, you use one flexible, ‘signposted’ questionnaire.

Reminder: any given questionnaire (= the data collection instrument) may use a combination of question types (= mix of closed and open ended questions).

 [3]  FORMAT

This relates to the structuring of the response categories of closed-ended questions.

The general format is to present all possible answers and have respondents choose between the different categories.

·        Note that the response categories offered should relate to the type of data being sought. Take the following example of a question to obtain data about respondent age:

Please indicate your age/ What age are you?

[   ]16 – 18                     [   ]19-24                       [   ] 25-34

[   ]35-44                       [   ]45-64                       [   ]65 or over

If the research objectives include description and exploration of the experiences of older people, then ’65 or over’ as above, may be too broad a category; instead you may want to merge some of the other response categories (as being data irrelevant to the research questions):

[   ]16 – 24                     [   ] 25-44

[   ]45-64                       [   ]65-74              [   ]75 or over

A ranking (or ordinal) scale may be used to structure responses to a question

e.g.

Q. People found in possession of cannabis for personal use should not receive jail sentences. Please indicate whether you:

1. Agree strongly

2. Agree

3. Neither agree nor disagree

4. Disagree

5. Disagree strongly

Other common response categories are:

1. Too little

2. About right

3. Too much

1. More

2. Same

3. Less

two general patterns:

A. FUNNEL (going from the general question to the specific question).

B. INVERTED FUNNEL

This sequence structure could apply to the entire questionnaire, or to specific topics / sections within the questionnaire.

In general, it’s advisable to start off with ‘objective’ descriptive and neutral/non-threatening questions and then move on to more ‘subjective’ and controversial questions.

The answers to the questions asked = the data on which answers to the research questions and objectives are built.

A fundamental requirement of the question is that it’s meaning should be clear to both the researcher and the respondent, at the data analysis stage and when the findings are being interpreted. If the questions - or any particular question - asked is unclear, then the validity and reliability of the research is suspect.

Example of an unclear question: “Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened?”

This was a question asked by the Roper polling organisation in the United States . On the basis of responses obtained, Roper concluded that one in five Americans thought the Holocaust never happened! (A subsequent survey carried out by Gallup found that about 3% of Americans doubted the Holocaust happened).