In a previous post we talked about the merits of a fox-like mentality in data science and how both mentalities are equally valid in general. Here we’ll talk about how either one can be equally catastrophic when used in its pure form. Because just like in life, extremes don’t play out well in data science, in spite of what the so-called experts lead us to believe.
Having a pure mentality may sound alluring, even noble, but this would only make sense in a perfect (or extremely imperfect) world. If we were living in a sci-fi utopia where everything was meticulously ordered and harmonious, then it wouldn’t make sense to try out anything outside the norm, since the norm itself would be the optimum course of action. We would be ruled by an extremely talented ruler (human or AI), and we wouldn’t need to think about what the best path of action would be since all of this would have been done for us beforehand. In such a case a hedgehog mentality would work best, since it would save us time and effort. On the other hand, if we were inhabiting a dystopian world where everything was corrupt, all references to tradition were lost or malformed, and chaos reigned supreme, then a fox mentality would work best because it would urge us to explore all possibilities and oftentimes create our own way of doing things. With such a mentality we would cleverly maneuver through the suboptimal strategies that were available until we found one that worked well.
However, unless you live in some country in South America or Eastern Europe, chances are that things aren’t that bad, while if you are not under the influence of some strong drug or some super-effective propaganda, you’ll see that things are not that great either. The world we live in is somewhere between the extremes that are so eloquently portrayed in the media and in the movies (though is generally worse than it is shown on YouTube). So, the question is why would someone want to employ a strictly fox-like or hedgehog-like mentality? Or, for the more skeptical people out there, what could possibly go wrong by sticking to our extreme mentality?
If you were to employ a strictly hedgehog mentality today, you’ll find yourself trapped in the worst kind of prison, one that would be practically impossible to escape from: your own thought bubble. You’d generally be a fanatic of sorts, swearing by this or the other methodology and shunning all other ways of thinking. That would be particularly bad if you are misinformed or ignorant to some extent, in which case you would favor you own suboptimal methods over other, more effective ones. A data scientist like that would probably find work only in academia, namely the most conservative universities (that serve some questionable agenda). Such a data scientist could also work in an established corporation, where group think is the norm and there is enough funding to maintain the essential illusions that would make such an existence tolerable. Finally, this kind of data scientists would also thrive in government offices where being able to articulate well what the managers want to hear is by far the most valuable skill, even more than actually building something that works well. A hedgehog data scientist today would be successful as a professional, since many people are still unaware of what data science really is and being able to speak those people’s extremely limited vocabulary (emphasizing all the essential buzzwords) can get you to a senior level position, even if you all you use is logistic regression. So, despite your external success, as a hedgehod data scientist today you’d be mediocre at best.
If you were to use a fox mentality in its purest form, things would be as bad. You’d probably find it impossible to get anything done, as you’d be pushed from one great idea to another with such force that you wouldn’t put enough time to a method for it to bear fruits. It would be much like navigating a sailboat through a harsh tempest. Unless you were living in a place of total chaos, you’d find it really hard to land a decent job and you wouldn’t be able to stick to one long enough to establish a career in a particular domain. The only places that would tolerate your attitude would be some very few universities that are committed to innovation, some research centers, and a certain kind of companies that value fringe science more than anything else. Most likely, you’d be looking at start-ups though, particularly the ones that rarely make it through the 2-year mark. A fox-like data scientist today would know all methods to some extent (and may even have a few of her own), but wouldn’t be the best person to convince someone to invest in any of them. Such a data scientist would be more invested in the science than the business aspect of the data science field, and although she would be adept in the field, she would enjoy very limited professional success.
Clearly in a world that is not too extreme, we need to have a combination of both mentalities (which is closer to the mentality of another forest creature that you are invited to find out). We ought to be hedgehogs when it comes to principles and strategy (the business aspect of things) and foxes when it comes to methods and tactics (the analytics part). A holistic data scientist appears fox-like to his fellow data scientists, as he doesn’t compromise with any method until he finds a truly great solution to a problem, but he is more like a hedgehog to other people, entirely committed to his craft and not caring so much about compensation and politics. Such a data scientist is rare but is definitely worth aspiring to and although he may not be what the world deserves right now, he is definitely what the world needs.
Zacharias Voulgaris, PhD
Passionate data scientist with a foxy approach to technology, particularly related to A.I.