Quality standards

SCOPE OF APPLICATION

1. This standard establishes basic requirements for the quality of marketing research in Ukraine.

TERMS AND DEFINITIONS

 2. The following definitions of terms are used in this standard:

2.1 Agency
A company that provides marketing research services.

2.2 Client
The company ordering marketing research.

2.3 Respondent
A person who has given consent to provide information for marketing research.

2.4 Research proposal
A document developed by the Agency and provided to the Client.

2.5 Field work
Work on the collection of primary data.

2.6 Reporting adjustment
Editing of the report at the request of the Client, if a reporting adjustment is applied that requires a return to the questionnaires.

2.7 Forced adjustment
Automatic control that performs sequential checking and correction of programs.

STANDARDS OF RESEARCH MANAGEMENT

General conditions of sale of research projects and indicators of their cost:

3. Research proposals for determining the price
Determination of the price for the implementation of a research project includes:

  • cost and calculation of research terms;
  • technical specification of the study;
  • reporting information – specification of the format in the received data, type of report, etc.;
  • the author of the proposal – for contact for the purpose of further discussion;
  • availability of certificates for compliance with quality standards, confidentiality of information data;
  • compliance with the requirements of current legislation in the research process.

A research proposal differs from a price determination in that, in addition to the plan listed above, after discussing the plan, the Client offers the best solution to the research problems posed.

4. Confidentiality

Strict confidentiality of all constituent parts of the research proposal is an ethical requirement that binds both parties – the Research Agency and the Client. First, the details of Client A’s needs must be kept strictly confidential by all Agencies potentially involved in the implementation of the research project. Second, unless otherwise specified, copyright in the research proposal, as well as all related documentation (questionnaire, discussion guide, etc.) shall remain the property of the Research Agency, and no detail of any proposal (order) shall to be disclosed, and even more so, to be provided to any third party.

5. Assumptions about the distribution of the research object

The research proposal must always indicate data on the level of prevalence of the research object, which are used by the parties when calculating the cost and terms of the research. The proposal should specify (or produce) the most optimal version of the proportion of the total population that is represented in the sample of the general population. The client is notified of all significant deviations from the initial assumptions that were made at an early stage of the project. Any changes in the cost or project execution terms are discussed and agreed upon with the Client.

6. Types of stimulation and tested goods

In the proposal, it is necessary to indicate specific types of incentives and goods (if needed) that are being tested, as well as their quantity. It should be noted whether the supply of these materials and goods is included in the proposed budget or not. This document should also contain information, however, whether the products should be returned to the Client after the completion of the project. In SOU 91.12.0/21708654/001/2002, the offers necessarily state that any product and (or) incentives provided by the Client must meet all legal requirements. The Client shall be responsible for any claims or legal actions that may arise as a result of the use of any goods and/or incentives provided by him in the study, and undertakes to release the Agency from all further liability in the event of such cases.

7. Duration of the interview

The research proposal should always include information on the duration of the interview (by the amount of time or by the number of questions) or group discussions, on the basis of which the cost was established. If the final version of the questionnaire or discussion guide required by the Client exceeds this duration, then the cost and timing of the project may be subject to revision.

8. Additional analysis

The research proposal should include an indication of how and to what extent the data will be analyzed, or the time frame for any additional analysis that may be provided as an anticipated component of the agreed contract. Beyond these limits, any analysis will be invoiced as an additional service not included in the main contract and paid for separately.

9. Contracts with subcontractors

The proposal must clearly indicate where subcontracting is planned for any significant part of the research project.

Basics of research:

10. Defining the goal and using the results

A research proposal should outline the purpose of the study and, where appropriate, indicate what marketing or other decisions may be affected by the results of the study.

11. Questionnaire or discussion topics

The research proposal should outline the topics to be covered by the questionnaire or discussion guide or other means of data collection. The exact wording of the questionnaire (discussion guide) should be determined only after the Client has officially approved the project.

12. Discussion of alternative methods

In that case, if it would be appropriate to use several types of data or several alternative methods of their collection to solve the problem, it is necessary to stipulate in the proposal the advantages and disadvantages of the alternative methods that can be used. It is also necessary to indicate the reasons on the basis of which the selective benefit of one of the approaches is proposed. When using a particular research technique (say, projective or similar techniques in qualitative research, or conjoint/analysis in quantitative research), its general characteristics should be clearly defined, along with the reasons why it is the best option. The choice of quantitative or qualitative methodology should also be justified when the circumstances require it.

Methods:

13. Method of primary data collection

The research proposal should contain a precise description of the recommended data collection method. Each of the necessary financial incentives should be thoroughly explained within the framework of the objectives and format of the study. If the approach is experimental or designed specifically for a given project (as opposed to a common, recognized method of data collection), then this should be clearly outlined.

14. Collection of secondary data

If the proposed method includes the collection or use of any secondary data, the proposal must specify:

  • source (statistical documentation, operational search, online search, etc.);
  • exact intended use.

Any known restrictions on the use of secondary data must be stipulated.

15. Research phase

The research proposal should contain a description of the purpose and method of the research phase, if any.

16. The ratio of private and corporate ownership

The proposal always indicates how the data will be collected – private or corporate, or which part of the results will be considered private and which corporate property, respectively.

Sample:

17. Target sample size

A research proposal always has or “must” contain the expected sample size for an effective survey. In the case of a multi-stage study, the minimum (or final) sample size must be determined. As for group discussions and individual in-depth interviews, it is necessary to indicate the number of people who will be interviewed and (or) the number of groups and the planned number of participants in each of them.

18. Definition of the general population

The proposal should clearly define which social groups are most suitable for the survey, as well as an accurate description of the criteria and procedures used in the process of selecting respondents.

19. Methods and methodology of sample research

The research proposal should indicate which sampling technique was chosen – random, pseudo-random, or quota. The expected number of survey points, the location of the interview and its description, the expected number of interviewers or moderators (for group discussions) and all other important parameters that affect the cost and quality of the research are indicated in the proposal, as well as the types of incentives incentives for respondents. All the criteria used for the selection (exclusion) of respondents (such as loyalty to a certain product, belonging to a certain advertising agency, etc.) must be determined in advance.

20. Selection of interview participants in families or organizations

When only one person is selected from a family or organization, the proposal must describe the procedure that will be used in the process of this selection.

In qualitative research, a minimum period of time must be specified since the participants last participated (if indeed participated) in a group discussion or individual in-depth interview. Proposals for such a study must also clearly define how respondents will be recruited (again, from a pre-compiled list or by other means).

Field work:

21. Method of instruction

The research proposal should contain a description of the instruction method used in work with foremen, interviewers and controllers. For example, meetings with all interviewers, only foremen, or sending instructions by mail, using e-mail, telephone, etc.

22. Place of interview

The research proposal should determine the location of the interview: on the street, at home, at work, in a special room, etc. If an agreement is reached with the respondents about the interview in advance, or if the research is carried out in several stages, then there may be several interview locations – at the same time, all the conditions and parameters of the specified types of interviews are specified. If the proposed location where the interview is to be held requires prior permission (eg in a shop) then this requirement should be clearly stated in the proposal. The day of the week, the time of the day when the interview will be held should also be determined.

In a qualitative study, the location of interviews and (or) group discussions (at home, in a hotel, in the office), the availability of visual aids, as well as the opportunity for the Client to attend a group or individual interview are also determined.

23. Planning work in the field

If there is a deadline for the results, or if the fieldwork or recruitment of respondents for interviews or participation in group discussions must be carried out by a specific date or on specific days or at a specific time of day, then the proposal should state the latest date by which certain consent must be obtained from the Client in order to meet the time requirements. The client must also be informed of the last date of receipt of goods or incentives to be used in the field work.

24. Quality control

At the Client’s request, the Agency must provide information on:

  • the method of checking the quality of data collection, which will be applied;
  • the proportion and type (or types) of interview verification, including engaging respondents for qualitative research;
  • type of data correction or cleaning;
  • all other measures used to minimize errors at all stages of the study.

Data processing and reporting:

25. Open questions

The expected number of open questions (questions open to different interpretations) should be reflected in the proposal, as should the method planned for the preparation of coding systems. The research proposal also indicates whether a list of responses to open-ended questions, as well as (or) frequency tables, will be provided.

26. Data weighting

The parameters of any data weighting method proposed in the study should be described in the proposal.

27. Report format

The proposal should indicate the format of the final results of the study: whether it is a full report, a report report, summary tables, or individual data files. They can be provided as documented on paper, or in electronic format, or both. All this should be reflected in the research proposal, and the number of copies of the report included in the price should be indicated.

The proposal for qualitative research must indicate whether transcripts of interviews and/or group discussions will be included in the report, as well as whether the presentation of the results is included in the total cost or paid for separately.

STANDARDS FOR IMPLEMENTATION OF RESEARCH PROJECTS

28. Research planning

After the approval of the project, the Agency’s primary task is to issue a plan (planning document), which shows the schedule of all important stages of the research:

  • commitment and date of supply of incentives and (or) material incentives, concepts and (or) goods for testing together with the agreement on supply;
  • the date of the Client’s approval of the questionnaire or group discussion guide or personal interview;
  • the obligation to obtain permission to use a certain place/point of survey;
  • schedule of field work;
  • the date of approval by the Client of the technical conditions of the analysis;
  • the earliest possible date for the submission of preliminary results;
  • possible dates for the report and presentation, if required.

The Agency must provide the Client with a contact person, as well as provide recommendations or suggestions for further actions in case of unforeseen circumstances.

29. Schedule of field work

At the request of the Client, the Agency must provide a complete schedule (date, time and location) of all fieldwork operations, including the briefing of interviewers, foremen or moderators. If necessary, the Agency must facilitate the presence of the Client at certain stages of field work, provided that:

  • does not lead to an unreasonable delay in field work operations;
  • does not interfere with the effective organization of field work, for example in the case of training or evaluation of interviewers;
  • does not jeopardize the correctness of data collection in case the research topic is sensitive;
  • does not endanger the anonymity of participants/respondents,
  • in the case of qualitative research, the presence of the Client or observation of groups is also regulated by subsection 53 of this standard.

Data collection and processing:

30. Registration of all field work operations

In order to make available all elements necessary for the analysis of all discrepancies between the intended and obtained sample structure, the Agency must keep records of:

  • of all sources used to construct the sample;
  • any interviewer quotas or random selection and analysis instructions;
  • all procedures used in recruiting respondents.

In order to fully clarify the differences between the intended and obtained samples, the following items should be recorded and analyzed, if necessary:

  • degree of prevalence and coverage levels;
  • level of “no”/answers;
  • in the case of product testing with one or more repeated interviews, the number of effective interviews achieved at each stage.

In the case of qualitative research, data will be stored for each respondent and the number of participants who actually participate in each group discussion.

31. Information records about interviewers

A file containing the following documented information shall be maintained for each field interviewer, foreman and controller, or telephone interviewer or foreman:

  • questionnaire;
  • date of interview or admission to training;
  • types, terms and duration of training received;
  • confirmation of previous training or experience, including knowledge of languages, practice of various types of interviews (structural, qualitative, business/interview, mystery shopping, etc.);
  • all received recommendations

On field interviewers:

  • accompanying report for the first job,
  • the validity check carried out in the last 12 months;

To the telephone interviewer:

  • a report on supervision or control carried out in previous works,
  • control monitoring of quality or verification of reliability for the last 12 months;

The types of projects they work on (product testing, business/interviewing, group recruitment, structural quantitative research, etc.);

  • attestation of previous activity;
  • details of significant deficiencies found in individual work and measures to correct them;
  • date of issue and ID number (if necessary).

32. Data protection

All fieldwork and data recording is conducted in accordance with data protection legislation. Neither the questionnaire, nor the audio/video recording from which the respondent can be identified, nor any other data record containing the respondent’s name, will be transferred to the Client or any third party, except if without the express consent of the respondent and for research purposes only. Records containing personal identification details of respondents should not be retained by the Agency for longer than is necessary for fieldwork review and subsequent quality checks and possible re-interviews.

If the recruitment of a respondent or participant involves the use of lists provided by the Client, then you/use of such lists must be subject to data protection legislation.

33. Anonymity of the client

The identity of the Client is not disclosed to respondents, except when the Research Agency and the Client jointly decide that there is good reason to do so.

34. Card of the respondent

At the end of any individual interview or group discussion, each respondent is given a document with thanks for cooperation and the name and address of the Research Agency, its phone number. When the interview is conducted by telephone, a return telephone number must be provided at the end of the interview.

35. Date and duration of the interview

The date and duration of the entire interview must be recorded. In the event that an interview or collective survey is recorded on audio/videotape, the recording must have a label with the recording date. The duration should be known from the recording.

36. Scope of the interviewer’s work

Fieldwork records should provide an opportunity to analyze the questionnaires collected by each interviewer.

37. Supervision and control of field work

The methods and ratio of verifications and control interviews (including respondents recruited for games/discussions) should be defined in the proposal and confirmed in the report. Each Agency has the right to establish its own monitoring and control standards with a defined minimum ratio of repeated calls, listening, or checks (monitoring) that are conducted between interviews. Fieldwork controllers, if necessary, analyze for each interviewer the time of day when the interview was conducted, the frequency per day and the “hit rate” (ratio of successful interviews per 10 calls).

In the case of qualitative research, the Agency must describe the system used to screen and control the recruitment of focus group participants or interviewees.

38. Installation of the data coding system

The Research Agency shall ensure that coders working in the project have been instructed or provided with instructions including the following:

  • general goals or stages of the research project;
  • identification of those issues that require coding;
  • specific subgroups needed to develop coding frameworks (eg, region, consumer/non-consumer, etc.);
  • any useful information from a previous project or stage;
  • any other requirement or special instructions specific to the project.

The Research Agency’s normal practice for coding frameworks for open questions shall be documented for the Client upon request. As part of the “No” coding, the answers always differ from the “I don’t know” answers. The “Other” category should not exceed 10% of respondents’ answers and should not lead to any distortions in the distribution of answers.

39. Data entry verification norms

The Research Agency must have documented data control procedures.

If checks on the validity of the information (consistency checks) are inadequate, other control procedures will be used, such as full or partial re-entry of data.

There are two types of response to detected errors:

  • making corrections to improve the quality of research;
  • development of appropriate instructions and/or actions to be taken against staff who made the mistake. In addition, it is necessary to establish the corresponding typical research minimum percentage for verification.

40. Correction or Purification of Data

At the Client’s request, the Agency must document the data correction/control method used after their entry (reporting or forced correction).

Reporting adjustment is a check back to questionnaires. When accounting adjustment is applied, the resulting decision must be recorded.

Forced correction is an automatic control that performs sequential checking and correction of programs. When a forced adjustment (often referred to as “data cleaning”) is used, the forcing logic must be documented and presented to the Customer for approval if desired, and a trial test must be performed with the results documented to ensure that that coercion has the desired effect.

For the smooth conduct of the study, in the event of the need to immediately process errors, rules are developed, which, in turn, are documented and applied immediately (so that the decisions made do not need to be reviewed or confirmed every time).

In all cases, the main data cleaning procedures and rules should be documented in the Technical Annex of the Research Report or be available upon request.

41. Data Cleansing File The Agency should pay attention to project files after data cleaning to ensure that:

  • files or data inside the file are not duplicated;
  • the latest version is used;
  • primary and cleaned data are clearly recognizable;
  • logic checks or equivalent procedures are performed before or after data cleaning.

Upon request, the Client is given access to such files according to his project.

Research report:

42. Completeness for reproducibility

A research report should always be detailed enough to allow any reader to replicate the study or perform a complete reanalysis. A description of what was investigated and what was found should be carefully documented. All methodological parameters, data collection techniques, material stimuli used, and analytical tools applied to the data should be described in detail.

In particular, the research report, if necessary, should include the following details:

  • research goals set at the beginning by the Client;
  • primary method(s) of data collection;
  • the general population and the method of reference to the general population;
  • levels of distribution and coverage (in quantitative research);
  • predicted and achieved sample sizes and structure;
  • raw and weighted base components of all calculated percentages;
  • significance level (90%, 95%, or 99%) of differences in results;
  • the exact questionnaire used in the field, including all material incentives, test items, rotating lists, etc. (in qualitative research, a discussion guide or personal interview);
  • after carrying out a multivariate analysis – a technical measure of significance.

If the Agency considers it appropriate, and depending on the research data and the type of research, these technical details will be placed:

  • in resume management,
  • except for each relevant table or text fragments in the body of the Report;
  • or in the technical appendices at the end of the Report. The following elements will also be included and must always be available at the Client’s request:
  • any secondary data used and its source;
  • essential parameters of the field work, including the period, frequency, geographical boundaries, tools for conducting a sample survey, days of the week when the interview was conducted, questionnaire for the survey, etc.;
  • any external fieldwork assistance, as well as any other subcontractors used for major parts of the study;
  • interview duration;
  • a general description of any incentives offered to respondents;
  • description of all weighting methods used by researchers.

43. Separation of interpretation from results

In both quantitative and qualitative studies, the interpretation of the results (conclusions and recommendations) in the Report should be clearly separated from the direct description of the data both when presented and when writing the report.

44. Validity of results

For many possible reasons, the results of any market research project may have some degree of uncertainty. The Agency’s ethical obligation is to clearly mention in the Report all identified sources of potential inaccuracy in the results.

For example, it will be very important to determine whether any clearly significant results are due to technical errors or are consistent with actual observations. For example, the sampling method could have led to a so-called “design effect,” or an external event could have occurred during the study that could seriously affect the significance of the results. In such cases, the Agency should identify and review the presence of such “noise” in the research results.

45. Definition of all variables

All variables used in data analysis should be clearly and completely defined. Any indexing, quantification, scaling, or grouping should be fully explained in a table showing all respondents’ grouping options. The client always has the right to see the tabulation of each question asked. Each of the component parts of any combined variable must be clearly identified to make the logic fully understandable to every reader. These technical details will appear:

  • if possible – in the relevant tables in the Report;
  • or as Technical Annexes to the Report. The only exception to the above occurs when the research analysis, model, or simulation/s are private and thus confidential.

46. ​​Data in electronic format

If the data is to be presented in electronic form, the format used must be agreed between the Client and the Agency in advance.

Management of subcontractors:

47. Transparency is necessary

Any anticipated subcontracting of any significant part of the research project must be mentioned in the research proposal. Any unforeseen changes to the project must be brought to the attention of the Client. It is the responsibility of the Agency to ensure that subcontractors and employees work to the same level of quality standards as themselves.

48. Full responsibility of the agency

The agency contracted by the Client to carry out the project is fully responsible for all work carried out in connection with the project, including any significant part of the work that may be subcontracted.

49. Other research agencies

The criteria for selecting other research agencies for subcontracting are:

  • experience, familiarity with the methodology or field related to this product, or experience with other agencies, etc.;
  • quality of service;
  • price (or value);
  • efficiency and transparency.

Membership in the Ukrainian Marketing Association or in ESOMAR and inclusion in the ESOMAR directory must take place after signing the ICC/ESOMAR International Procedural Code of Marketing and Sociological Research, as required by these standards. It is not mandatory, but it is recommended that agencies are certified for compliance with ISO 9000. An agency that works according to the EFAMRO Marketing Research Quality Standards or according to these standards has the right to expect the same level of quality from subcontractors.

SPECIFIC TYPES OF MARKETING RESEARCH

International studies:

50. Coordination of international projects

An agency authorized to coordinate an international research project must be bound by the same obligations of transparency and accountability as in the case of a single-country project. In the case of international research, project planning (see subsections 28 and 29), transparency of subcontracting (subsection 47) and Agency accountability (subsection 48) are even more important to ensure the highest quality for the Client.

The client of the international project must be informed about:

  • distribution of tasks between the coordinating Agency and local subcontractors in different countries;
  • how interviewers, recruiters and moderators should be instructed and supervised in each country;
  • rules applicable to data encoding, data entry, verification and cleaning in each country;
  • language capabilities of the Coordinating Agency in the local languages ​​that will be used in the study;
  • how the Coordinating Agency maintains communication with local subcontractors during the project. All these issues are clarified in the proposal or in the plan issued by the Coordinating Agency.

51. Translations

The agency should specify how questionnaires, discussion guides, and other documents will be translated and adapted as necessary, and how translations will be checked.

All translators must be native speakers of at least one of the languages ​​used. If their mother tongue is only one language, then they must also be fluent (at least for several years) in other languages ​​used.

Qualitative research:

52. Training of interviewers and moderators

Each moderator gets acquainted directly with the purpose of the research, research questions, material incentives and specific methodology before starting field work. This requirement also applies to material incentives or any other materials beyond the Agency’s control.

53. Observation of the Client on group discussions and interviews

All Agencies ensure that the recording and observation of fieldwork for qualitative research is consistent with the requirements of current national legislation and the content of the European Society for Opinion and Marketing Research (ESOMAR) Directives on tape and video recording, Client observation of interviews and group discussions.

The client or his representatives can watch the interview or group discussion through a one-way mirror or cable TV, but the participants must be informed and agree to this at the beginning of the session. Sufficient evidence of the fulfillment of this condition is a statement on this issue, which is included and recorded at the beginning of each group discussion or interview.

If there is a possibility that the Client can identify any of the participants (as a personal acquaintance or as a member of an organization), then the Client is not allowed to view the session.

All Clients observing individual interviews or group discussions must sign a document stating that any information they obtain while observing the interview or group discussion will be used solely for the purposes of the relevant research project. They must also agree to comply with the requirements of the International Code of Conduct for Marketing and Social Research of the International Chamber of Commerce of the European Society of Opinion and Marketing Research (ICC/ESOMAR), current national legislation and must obtain copies of the specified documents/documents, if they do not already have them.

54. Registration (record)

According to the instructions in Rule 7 of the International Procedural Code of Marketing and Social Research of the International Chamber of Commerce of the European Society of Opinion and Marketing Research (ICC/ESOMAR), respondents are always informed that they are being recorded on audio or videotape. It is also necessary to check the compliance of the actions with the requirements of subsections 1 to 7 of the “ESOMAR Directives on audio and video recording and client observation of interviews and group discussions”, as well as other standards of national professional codes and directives.

55. Qualifications of group moderators and interviewers

At the Client’s request, the Research Agency specifies the experience and qualifications of discussion group moderators and interviewers for in-depth interviews.

56. Use of employees, hired interviewers and independent group moderators

Where independent group moderators, subcontractors, or employees (i.e., individuals not employed by the research Agency) are used in connection with the project, the Agency must include a confidentiality clause in the contract with such individuals regarding information they may receive during the course of the project . Upon request, the Client shall be provided with a full description of the qualifications and experience of all subcontractors, hired workers or group moderators working on the project.