Critical thoughts about big data analysis
Completed on 02-Jul-2016 (57 days)
I am a senior programmer and numerical modeller with relevant experience in data-intensive software developments. In fact, my professional career as a programmer did precisely begin within a data-modelling-intensive environment. To know more about my theoretical and practical background, visit the about page of this site
Nevertheless, most of my data modelling expertise has been focused on the adequate understanding of small amounts of information, rather than on dealing with as big as required datasets. Modelling the underlying phenomenon by means of analysing small-but-descriptive sets of data rarely involves dealing with the typical big-data concerns, as described in the corresponding section
. Thus and purely speaking, I have a limited big-data forecasting experience.
During the last weeks, I have been participating in various big-data problems and open challenges. I have confirmed the expected differences between small-high-quality and big-random-quality data models and learned quite a lot from these short-but-intense episodes. In fact, I am including my impressions, learned-lessons and recommendations about how to face big-data problems in one of the last sections
and the appendix
of this project.