Comparative Analysis of Regression Methods in Small-Sample Agricultural Time Series: The Case of Turkish Beekeeping Data
DOI:
https://doi.org/10.5281/zenodo.22737079Abstract
Abstract: Accurate modeling of agricultural time series with a limited number of observations is a significant challenge, as multicollinearity and small sample sizes may reduce the reliability of conventional regression methods. In this study, the predictive performance of five different regression techniques was compared for modeling real honey prices using Turkish beekeeping data covering the period 2015–2025. Ordinary Least Squares (OLS) regression, Ridge Regression, Elastic Net Regression, Principal Component Regression (PCR), and Partial Least Squares Regression (PLSR) were evaluated using a nested leave-one-out cross-validation approach. Model performance was compared based on the coefficient of determination (R²), root mean square error (RMSE), mean absolute error (MAE), and predicted residual sum of squares (PRESS). Among the evaluated models, PLSR demonstrated the highest predictive performance under cross-validation, with an R² of 0.972, RMSE of 1.937, MAE of 1.563, and PRESS of 41.290. Correlation analysis revealed substantial multicollinearity among the explanatory variables, supporting the use of latent-variable-based regression methods. The findings indicate that PLSR provides a robust and reliable approach for modeling small-sample agricultural time series and represents a potentially applicable method for agricultural price forecasting and decision-making processes in the beekeeping sector.
References
Agarwal, A., Shah, D., Shen, D., & Song, D. (2019). On robustness of principal component regression. Advances in Neural Information Processing Systems, 32.
Aslam, M. (2014). Using heteroscedasticity-consistent standard errors for the linear regression model with correlated regressors. Communications in Statistics—Simulation and Computation, 43(10), 2353–2373.
Avinash, G., Ramasubramanian, V., Ray, M., Paul, R. K., Godara, S., Nayak, G. H. H., Kumar, R. R., Manjunatha, B., Dahiya, S., & Iquebal, M. A. (2024). Hidden Markov guided deep learning models for forecasting highly volatile agricultural commodity prices. Applied Soft Computing, 158, 111557. https://doi.org/10.1016/j.asoc.2024.111557
Bal, A. (2024). Improving the robustness of the Theil-Sen estimator using a simple heuristic-based modification. Symmetry, 16(6), 698. https://doi.org/10.3390/sym16060698
Boggy, G. J., & Woolf, P. J. (2010). A mechanistic model of PCR for accurate quantification of quantitative PCR data. PLOS One, 5(8), e12355.
Burton, A. L. (2021). OLS (linear) regression. In The Encyclopedia of Research Methods in Criminology and Criminal Justice (Vol. 2, pp. 509–514).
Cawley, G. C., & Talbot, N. L. C. (2006). Gene selection in cancer classification using sparse logistic regression with Bayesian regularization. Bioinformatics, 22(19), 2348–2355. https://doi.org/10.1093/bioinformatics/btl386
Cheng, J. H., & Sun, D. W. (2017). Partial least squares regression (PLSR) applied to NIR and HSI spectral data modeling to predict chemical properties of fish muscle. Food Engineering Reviews, 9(1), 36–49. https://doi.org/10.1007/s12393-016-9147-1
Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carré, G., García Marquéz, J. R., Gruber, B., Lafourcade, B., Leitão, P. J., Münkemüller, T., McClean, C., Osborne, P. E., Reineking, B., Schröder, B., Skidmore, A. K., Zurell, D., & Lautenbach, S. (2013). Collinearity: A review of methods to deal with it and a simulation study evaluating their performance. Ecography, 36(1), 27–46. https://doi.org/10.1111/j.1600-0587.2012.07348.x
Faraz, A., Tırınk, C., Önder, H., Şen, U., Ishaq, H. M., Tauqir, N. A., Waheed, A., & Nabeel, M. S. (2023). Usage of the XGBoost and MARS algorithms for predicting body weight in Kajli sheep breed. Tropical Animal Health and Production, 55(4), 276. https://doi.org/10.1007/s11250-023-03700-6
Gadedjisso-Tossou, A., Adjegan, K. I., & Kablan, A. K. M. (2021). Rainfall and temperature trend analysis by Mann–Kendall test and significance for rainfed cereal yields in Northern Togo. Sci, 3(1), 17. https://doi.org/10.3390/sci3010017
Ghasemzadeh, H., Hillman, R. E., & Mehta, D. D. (2024). Toward generalizable machine learning models in speech, language, and hearing sciences: Estimating sample size and reducing overfitting. Journal of Speech, Language, and Hearing Research, 67(3), 753–781. https://doi.org/10.1044/2023_JSLHR-23-00273
Guindani, L. G., Oliveira, G. A., & Ribeiro, M. H. D. M. (2024). Exploring current trends in agricultural commodities forecasting methods through text mining: Developments in statistical and artificial intelligence methods. Heliyon, 10(23), e40568. https://doi.org/10.1016/j.heliyon.2024.e40568
Hastie, T., Montanari, A., Rosset, S., & Tibshirani, R. J. (2022). Surprises in high-dimensional ridgeless least squares interpolation. The Annals of Statistics, 50(2), 949–986.
Hastie, T., Tibshirani, R., & Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd ed.). Springer.
Helland, I. S. (1990). Partial least squares regression and statistical models. Scandinavian Journal of Statistics, 17(2), 97–114.
Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55–67. https://doi.org/10.1080/00401706.1970.10488634
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). An Introduction to Statistical Learning: With Applications in R (2nd ed.). Springer.
Jin, Y., Li, W., & Gil, J. M. (2024). Forecasting fish prices with an artificial neural network model during the tuna fraud. Journal of Agriculture and Food Research, 18, 101340. https://doi.org/10.1016/j.jafr.2024.101340
Jolliffe, I. T. (1982). A note on the use of principal components in regression. Applied Statistics, 31(3), 300–303. https://doi.org/10.2307/2348005
Klein, N. (2024). Distributional regression for data analysis. Annual Review of Statistics and Its Application, 11, 321–346. https://doi.org/10.1146/annurev-statistics-040722-053607
Kyriazos, T., & Poga, M. (2023). Dealing with multicollinearity in factor analysis: The problem, detections, and solutions. Open Journal of Statistics, 13(3), 404–424.
Lee, H., Park, Y. M., & Lee, S. (2015). Principal component regression by principal component selection. Communications for Statistical Applications and Methods, 22(2), 173–180.
Mahaboob, B., Venkateswarlu, B., Narayana, C., & Balasiddamuni, P. (2018). A treatise on ordinary least squares estimation of parameters of linear model. International Journal of Engineering & Technology, 7(4.10), 518.
Mehmood, T., Liland, K. H., Snipen, L., & Sæbø, S. (2012). A review of variable selection methods in partial least squares regression. Chemometrics and Intelligent Laboratory Systems, 118, 62–69. https://doi.org/10.1016/j.chemolab.2012.07.010
Meshram, S. G., Kahya, E., Meshram, C., Ghorbani, M. A., Ambade, B., & Mirabbasi, R. (2020). Long-term temperature trend analysis associated with agriculture crops. Theoretical and Applied Climatology, 140(3–4), 1139–1159. https://doi.org/10.1007/s00704-020-03137-z
Mevik, B. H., & Wehrens, R. (2007). The pls package: Principal component and partial least squares regression in R. Journal of Statistical Software, 18(2), 1–23. https://doi.org/10.18637/jss.v018.i02
Moscovich, A., & Rosset, S. (2022). On the cross-validation bias due to unsupervised preprocessing. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(4), 1474–1502. https://doi.org/10.1111/rssb.12537
Önder, H., Tirink, C., Yakubets, T., Getya, A., Matvieiev, M., Kononenko, R., Şen, U., Özkan, Ç. Ö., Tolun, T., & Kaya, F. (2025). Predicting live weight for female rabbits of meat crosses from body measurements using LightGBM, XGBoost and support vector machine algorithms. Veterinary Medicine and Science, 11(1), e70149. https://doi.org/10.1002/vms3.70149
Owen, A. B. (2007). A robust hybrid of lasso and ridge regression. Contemporary Mathematics, 443, 59–72.
Parvandeh, S., Yeh, H. W., Paulus, M. P., & McKinney, B. A. (2020). Consensus features nested cross-validation. Bioinformatics, 36(10), 3093–3098. https://doi.org/10.1093/bioinformatics/btaa046
Pavlou, M., Omar, R. Z., & Ambler, G. (2024). Penalized regression methods with modified cross-validation and bootstrap tuning produce better prediction models. Biometrical Journal, 66(5), e202300245. https://doi.org/10.1002/bimj.202300245
Şahin, M. (2026). Estimate of albumen height in Atak-S hen eggs using ridge regression. Black Sea Journal of Agriculture, 9(4), 520–525.
Schreiber-Gregory, D. N. (2018). Ridge regression and multicollinearity: An in-depth review. Model Assisted Statistics and Applications, 13(4), 359–365. https://doi.org/10.3233/MAS-180446
Sen, P. K. (1968). Estimates of the regression coefficient based on Kendall's tau. Journal of the American Statistical Association, 63(324), 1379–1389. https://doi.org/10.1080/01621459.1968.10480934
Sharma, P., Villegas-Diaz, R., & Fennell, A. (2024). Predicting grapevine physiological parameters using hyperspectral remote sensing integrated with hybrid convolutional neural network and ensemble stacked regression. Remote Sensing, 16(14), 2626. https://doi.org/10.3390/rs16142626
Šverko, Z., Vrankić, M., Vlahinić, S., & Rogelj, P. (2022). Complex Pearson correlation coefficient for EEG connectivity analysis. Sensors, 22(4), 1477. https://doi.org/10.3390/s22041477
Tilevik, A. (2026). PCR, PLS, and Lasso Regression. In Multivariate Statistics and Machine Learning in R for Beginners: With Applications in Biology and Medicine (pp. 279–297). Springer Nature Switzerland.
Tirink, C., & Önder, H. (2022). Comparison of M, MM and LTS estimators in linear regression in the presence of outlier. Turkish Journal of Veterinary & Animal Sciences, 46(3), 420–428.
Vabalas, A., Gowen, E., Poliakoff, E., & Casson, A. J. (2019). Machine learning algorithm validation with a limited sample size. PLOS One, 14(11), e0224365. https://doi.org/10.1371/journal.pone.0224365
Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics, 7, 91. https://doi.org/10.1186/1471-2105-7-91
Wainer, J., & Cawley, G. C. (2021). Nested cross-validation when selecting classifiers is overzealous for most practical applications. Expert Systems with Applications, 182, 115222. https://doi.org/10.1016/j.eswa.2021.115222
Wang, X., Wang, X., Ma, B., Li, Q., & Shi, Y. Q. (2021). High precision error prediction algorithm based on ridge regression predictor for reversible data hiding. IEEE Signal Processing Letters, 28, 1125–1129.
Wold, S., Ruhe, A., Wold, H., & Dunn, W. J., III. (1984). The collinearity problem in linear regression: The partial least squares (PLS) approach to generalized inverses. SIAM Journal on Scientific and Statistical Computing, 5(3), 735–743. https://doi.org/10.1137/0905052
Xu, J., & Hsu, D. J. (2019). On the number of variables to use in principal component regression. Advances in Neural Information Processing Systems, 32.
Yates, L. A., Aandahl, Z., Richards, S. A., & Brook, B. W. (2023). Cross validation for model selection: A review with examples from ecology. Ecological Monographs, 93(1), e1557. https://doi.org/10.1002/ecm.1557
Yue, S., & Pilon, P. (2004). A comparison of the power of the t test, Mann-Kendall and bootstrap tests for trend detection. Hydrological Sciences Journal, 49(1), 21–37. https://doi.org/10.1623/hysj.49.1.21.53996
Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2), 301–320. https://doi.org/10.1111/j.1467-9868.2005.00503.x
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Black Sea Journal of Statistics
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
