Market Research Analysis Methods and Validation
Work Principles
First read Sources and Rules to confirm the book manuscript basis and rule scope. For deadlines, page counts, weights, anonymity, AI, and submission methods, verify the official documents for the target edition, category, and stage online; if verification fails, mark as "to be verified" but proceed with research content. Do not treat book manuscript examples as hard-and-fast competition rules.
Default language: Chinese. First review the user's existing materials and current progress; only ask for missing information that would change the plan; state assumptions for the rest and continue. When no raw data is available, provide only plans, fillable templates, or clearly labeled demonstrations; do not fabricate samples, statistics, interviews, references, or awarded identities. Deliver according to the depth of the user's task; local modifications do not require redoing the entire workflow.
Objectives
Based on Chapters 3, 5, and 6 of the book manuscript, bind each method to the research question, data conditions, and interpretable output. First determine whether the task is description, comparison, explanation, clustering, prediction, or concept exploration, then select the model.
Gateway for Data Entering Analysis
- Confirm the unit of analysis, observation period, sample source, probability/non-probability design, strata/clusters/weights, repeated measures, questionnaire version, inclusion/exclusion rules, and data dictionary. Without a complete sampling frame and known selection mechanism, do not claim simple random sampling.
- Keep raw data read-only; separate data cleaning, analysis, and output layers. Check fields and item numbers, recoding, reverse scoring, multiple-choice expansion, skip patterns, duplicates, logical conflicts, and missing data mechanisms. Distinguish not applicable, refused, don't know, and system missing.
- Manually review anomalies with multiple pieces of evidence; do not delete samples solely based on duration, same IP, or consecutive identical responses. Log rules, evidence, hit counts, actions, and versions; compare the impact of key cleaning rules on conclusions.
- Report the flow of contacts, responses, excluded and analyzed samples. When models have different N due to missing data, explain each separately. Do not repeatedly modify groups, delete items, or filter samples for significance.
Method Decision Table
| Task/Data | Available Methods | Pre-Checks | Main Deliverables & Limitations |
|---|---|---|---|
| Status & Distribution | Frequencies, proportions, mean/median, dispersion, and intervals | Measurement level, skewness, denominator, weights, sample coverage | Sample profile and boundaries; large sample size alone does not imply representativeness |
| Categorical Variable Association | Cross-tabulation, chi-square or exact tests; complex design uses corrected variants | Independence, sparse cells, repeated observations | Group proportions, association measures with intervals, not causal |
| Continuous/Ordinal Indicator Differences | t/Welch, ANOVA or appropriate rank tests; paired uses paired methods | Independent or paired, distribution, heteroscedasticity, ordinal meaning | Difference magnitude, effect size with interval; rank tests should not be vaguely called mean tests |
| Multi-Item Latent Dimensions | EFA; with a priori structure, CFA | Construct semantics, item correlations, distribution, sample size and estimator | Dimensions, loadings, and measurement evidence; EFA/CFA on same sample is not independent validation |
| User Segmentation | Clustering/mixture models appropriate for variable type | Scaling, outliers, redundant variables, distance, weights, and channels | Stable clusters, external profiles, reach rules; do not treat cluster names as fixed personality types |
| Continuous/Binary/Ordinal/Multinomial/Count Outcomes | Linear, Logistic, Ordinal or Multinomial, Poisson/Negative Binomial, etc. | Coding reference, outcome distribution, collinearity, residuals or overdispersion, hierarchical dependence | Coefficients/odds ratios/probabilities or count ratios with intervals; conditional association ≠ causation |
| Theoretical Latent Variable Paths | SEM and indirect effect estimation | First evaluate measurement model, identification, sample, estimator, fit, and competing models | Path and indirect effect intervals; cross-sectional mediation does not prove transmission mechanisms |
| Time Series | Decomposition, appropriate time series models, rolling forecasts | Time granularity, trend/seasonality, autocorrelation, anomalies and structural changes | Time-based holdout with simple baseline, prediction intervals; do not randomly shuffle time series |
| Concept Exploration for New Phenomena | Interview content/thematic analysis; grounded theory when design fits | Recruitment, audio transcription, consent, coding chain, and negative cases | Quotes → codes → categories → interpretation; saturation with evidence; no iterative theoretical sampling → not packaged as full grounded theory |
| Platform Reviews & Open-Ended Questions | Manual content analysis, word frequency/co-occurrence, topic/sentiment models | Source coverage, deduplication, domain lexicon, labels, and human verification | Representative texts, stability, validation error; word clouds do not substitute analysis; comment frequency ≠ population rate |
| Prediction & Machine Learning | Baselines and predictive models matched to target and sample | Data leakage, class imbalance, group/time split | Independent test performance, calibration/error and group performance; feature importance ≠ causal effect |
Key Judgments
Sampling: Determine sample size based on target precision, effect size, subgroups, model complexity, design effect, and nonresponse; write assumptions clearly. Probability sample formulas cannot justify precision for convenience samples. Complex design analyses must use strata, clusters, weights, and corresponding variance estimation. For weighted analysis, report unweighted N, weight source, distribution, and assumptions; do not claim weighting eliminates self-selection bias.
Measurement: First check reverse-coded items and construct coverage, then evaluate reliability and validity. α is not mandatory for all questionnaires and does not prove unidimensionality or validity; do not mechanically compute α for demographic items and formative indicators. For EFA, use parallel analysis/scree plot, semantics, and rotation; for CFA, report the estimator and appropriate fit indices, and do not substitute a single threshold pass for judgment. When measurement is unstable, stop interpreting SEM structural paths; either revise the design or fall back to exploration, description, and qualitative evidence.
Comparison: Simultaneously report direction, magnitude, uncertainty, and practical significance. For multiple groups and multiple tests, specify the multiple comparison strategy; "significant in one group but not another" does not prove a group difference in effects; directly test interaction or contrast. For cross-group scale comparisons, pay attention to measurement invariance.
Model: First write variables and model specification rationale, then estimate and diagnose, and finally interpret; do not select results by repeatedly trying until significant. Logistic odds ratios should not be interpreted as risk ratios; standardized coefficients should not be reported as raw unit changes. Robust standard errors do not fix omitted variable bias. Good fit and many control variables do not prove causation; without credible identification, use terms like "correlation/prediction/mechanism clues."
Prediction: Divide training/validation/test sets; fit preprocessing and feature selection only on training folds. Group records from the same subject/entity, and isolate time periods for time series. Compare to a naive baseline. For classification, use precision, recall, F1, confusion matrix, discrimination, and calibration as appropriate for the task; do not list all as decoration. With small samples, do not force machine learning for complexity's sake.
Qualitative & Text: Document the codebook, coding examples, and review/dispute resolution. For automated classification or extraction, create a representative human-validated set and report error rates and applicable context. When integrating multiple sources, align units, time, and concepts before merging; do not count items as individuals or arbitrarily concatenate personal records.
Output and Validation
First deliver an analysis plan: Question | Variable types and coding | Sample/design | Methods and selection rationale | Diagnostic assumptions | Result format | Validation and alternatives | What cannot be answered. If real data and execution conditions exist, then provide a data quality summary, reproducible syntax or code, core tables and figures with interpretation, robustness checks, and limitations.
Freeze data, record software version and random seed; ensure code and figures correspond to the same version. Conduct sensitivity checks based on actual risks: missing data, weights, outlier rules, specifications, cluster stability, etc.; do not mechanically pile up tests. Distinguish planned, revised, and exploratory analyses, and report non-significant results. For software output that cannot be executed, only interpret visible content; do not claim replication.
When complete, answer: What evidence supports each question? Are assumptions met? Under what conditions do results change? What actions can be supported? When data is insufficient, provide downgraded plans or requirements for additional data collection.