Methodology
The following rules apply essentially to surveys where results are projected to the entire population under study. A failure to comply with any of the following standards must be explained in the report by the author.
Sample
The sample will be representative of the whole universe on which the data will be projected. It will be sufficiently large to meet the precision levels stated in the study objectives of the research proposal (see confidence level table).
Weighting of results
The weighting coefficient of any single answer will not exceed three, excluding the weighting that may have been already applied by household.
Response rate
All reasonable efforts will be made to maximize response rates. As a guideline, a response rate of not less than 50 % will be aimed at for a telephone survey on a random proportional probabilistic sample of Québec residents age 18 and over. This response rate will be calculated as per the AIRMS standards appended.
Validation
Where no supervision is provided on-site, a minimum of 10 % of each interviewer’s interviews will be validated.
Where direct on-site supervision is feasible, a minimum of 10 % of all interviews will be directly validated. Proof of such validation will be kept for a minimum of six months.
In the case that the interviews of an interviewer would not be validated, 100 % of the interviewer’s work must be rejected or replaced.
The method of validation, the proportion of work validated, as well as the personnel responsible for the validation must be indicated in the report.
Report
The report must include :
- A complete description of the sample, including its design, execution, and weighting, will be presented in order that the reader be in a position to judge if the results are reasonably representative of the universe under study (including the weighting factors).
- The results of the last contact with the respondent or household. The results will be presented in accordance with AIRMS standards in the administrative report of contacts and response rates (see appendix).
- A copy of the questionnaire used for the survey.
- A copy or a description of all other exhibits showed or used to conduct the interview.
- A description of all sources of secondary data used.
- The research firm must provide to the client : when, how, and where the interviews were conducted; who has executed the data entry, and the data processing.
Subcontracting
Validation and procedure shall always be monitored by a AIRMS member unless otherwise agreed on with the client, in which case a failure to conform to AIRMS standards shall be clearly emphasized in the report.
Interviewer identification
Personal interviews
The interviewer must carry the identification of the company with a time expiry date.
Telephone interviews
In the case of telephone interviews, the identity of the research firm will be given in the introductory statement on all interviews.
Recording of interviews
The audio or audiovisual recording of interviews is not permitted without the knowledge of respondents. If a discussion facility includes a one-way mirror, participants must be told that clients sometimes observe these discussions.
Confidentiality
We assure complete anonymity to respondents and confidentiality of their specific answers. We assure complete confidentiality of the study and of the results to clients.
Storage of documents
The research firm shall be entitled to destroy original questionnaires 12 months after the end of fieldwork, and other data forms 24 months after the end of field work without prior notice to the client.
If the client wishes exceptions to this, he or she must make special arrangements with the research firm.
Publication standards for reporting of survey methodology
The fundamental objective of AIRMS is to ensure the highest methodological standards in the practice of surveys.
The following are the 13 points that AIRMS considers as the strict minimum that should be published on the methodology of any survey reported by the written press :
- The name of the research firm responsible for the survey. If the fieldwork was executed by a sub-contractor or by another entity, the name of such should also be published.
- The name of the project manager as well as his or her status in the firm.
- The name of the client for which the survey was conducted, as well as the department within this company that has ordered the survey.
- The description of the population under study including all the relevant specifications (i.e. adults, Canadian citizenship, language, regions or territories, etc.)
- The sampling method and the type of sampling (original sample frame, selection process of households and individuals, number of callbacks, with or without substitution, with or without stratification, type of stratification).
- The rate of response calculated as per AIRMS standards.
- A control of the representativeness of the weighted sample (sex, age, region, language and other relevant variables).
- The weighting procedure, where applicable (i.e. regional adjustment, weighting on the size of the household, etc.).
- The survey method. If by telephone, the number and location of the call centers, the proportion of work validated and the type of validation will be disclosed. If interviewing was computer-assisted, it will be mentioned and the proportions of calls monitored indicated.
- The interviewing period.
- The duration of a typical interview and a list of the main sections of the questionnaire as well as their logical sequence.
- The maximum margin of error based on a 95 % confidence level.
- The relevant warnings indicated in the report by the researcher (i.e. a vote intention is not a forecast, etc.).
The methodology section should be presented in an inset, separately from the analysis of the results.
Confidence level table
Level of confidence : 95 % (Unweighted random sample)
| Size of sample | Margin of error for a result of: |
| 2 % or 98 % |
4 % or 96 % |
6 % or 94 % |
8 % or 92 % |
10 % or 90 % |
12 % or 88 % |
15 % or 85 % |
20 % or 80 % |
25 % or 75 % |
30 % or 70 % |
35 % or 65 % |
40 % or 60 % |
45 % or 55 % |
50 % | |
| 100 | 3,8 | 4,7 | 5,3 | 5,9 | 6,4 | 7,0 | 7,8 | 8,5 | 9,0 | 9,3 | 9,6 | 9,8 | 9,8 | |
| 150 | 3,1 | 3,8 | 4,3 | 4,8 | 5,2 | 5,7 | 6,4 | 6,9 | 7,3 | 7,6 | 7,8 | 8,0 | 8,0 | |
| 200 | 2,7 | 3,3 | 3,8 | 4,2 | 4,5 | 4,9 | 5,5 | 6,0 | 6,4 | 6,6 | 6,8 | 6,9 | 6,9 | |
| 250 | 1,7 | 2,4 | 2,9 | 3,4 | 3,7 | 4,0 | 4,4 | 5,0 | 5,4 | 5,7 | 5,9 | 6,1 | 6,2 | 6,2 |
| 300 | 1,6 | 2,2 | 2,7 | 3,1 | 3,4 | 3,7 | 4,0 | 4,5 | 4,9 | 5,2 | 5,4 | 5,5 | 5,6 | 5,7 |
| 400 | 1,4 | 1,9 | 2,3 | 2,7 | 2,9 | 3,2 | 3,5 | 3,9 | 4,2 | 4,5 | 4,7 | 4,8 | 4,9 | 4,9 |
| 500 | 1,2 | 1,7 | 2,1 | 2,4 | 2,6 | 2,8 | 3,1 | 3,5 | 3,8 | 4,0 | 4,2 | 4,3 | 4,4 | 4,4 |
| 600 | 1,1 | 1,6 | 1,9 | 2,2 | 2,4 | 2,6 | 2,9 | 3,2 | 3,5 | 3,7 | 3,8 | 3,9 | 4,0 | 4,0 |
| 800 | ,97 | 1,4 | 1,6 | 1,9 | 2,1 | 2,3 | 2,5 | 2,8 | 3,0 | 3,2 | 3,3 | 3,3 | 3,4 | 3,5 |
| 1000 | ,87 | 1,2 | 1,5 | 1,7 | 1,9 | 2,0 | 2,2 | 2,5 | 2,7 | 2,8 | 3,0 | 3,0 | 3,1 | 3,1 |
| 1200 | ,79 | 1,1 | 1,3 | 1,5 | 1,7 | 1,8 | 2,0 | 2,3 | 2,5 | 2,6 | 2,7 | 2,8 | 2,8 | 2,8 |
| 1500 | ,71 | 1,0 | 1,2 | 1,4 | 1,5 | 1,6 | 1,8 | 2,0 | 2,2 | 2,3 | 2,4 | 2,5 | 2,5 | 2,5 |
| 2000 | ,61 | ,86 | 1,0 | 1,2 | 1,3 | 1,4 | 1,6 | 1,7 | 1,9 | 2,0 | 2,1 | 2,1 | 2,2 | 2,2 |
Key to interpretation : for a sample of 500, a result of 4% will be accurate to plus or minus 1.7 percentage points, 19 times out of 20. |
Administrative report of contacts and response rate (1)
| Numbers generated | FREQUENCY | Total |
|
||||
|
||||
|
||||
|
||||
|
| Calculation of response rate |
 |
(1) Presented as per AIRMS standards. (2) For a number to be considered “No answer”, there must be no answer at the number throughout period of data collection. For example, an appointment for which there is no answer at the time it is called back must be considered “Call back not completed” and not “No answer”.
|
