Getting the real deal: Why data mining is important

By Deepak GhodkeBig data is the key to a business’s success, big data will change the world, and big data will do this and that. How many times have we heard statements like these? Well I’ve lost count!! Big data and analytics are certainly the buzzwords today and the statements around big data being the differentiator between a successful business and a not so successful one are absolutely true, no denying that.I might have started the article in a way that might have given you an impression that I’m not a big fan of big data well I certainly am not a fan or let me say not just a fan but I’m a propagator of the technology. However, having said that I do not intend to talk about the importance or meaning of big data and analytics because there already has been a lot of talk around this.[caption id="attachment_1998265" align="alignleft" width="380"] Pic[/caption]What I intend to do is deep dive into one particular aspect of data analytics i.e. data mining. Big data and analytics is this one huge umbrella under which you will find a lot of other technologies/methods/concepts so to speak and data mining is one of those.Today the amount of data being generated around the globe every single minute is humungous. When we sit down to look at this tonnes and tonnes of data and understand the way it is being used, it’s imperative that we understand the power of data analytics. If we make the data we have available to somebody else without data mining being in the picture all that somebody knows is what we have told them. Now, if we bring data mining back in the picture then not only does the third person know what we have told him/her but can also guess a great deal more. Simply put data mining enables a business to use the information provided by a customer to reveal more than the customer could ever imagine.To most, data mining is where tonnes of data is collected and then data scientists/wizards work a spell on it and the data starts talking and reveals things nobody ever thought of. But how? Well for the most part, data mining tells us about very large and complex data sets, kind of information that will be readily apparent about small and simple things. However, a task that seems quite simple with 5 or 6 data-points is not that simple with data-points that run into billions.Today the amount of data that’s being generated is far more than we can handle, almost every single activity or interaction leaves a trail that somebody somewhere captures, stores and analyses. Just the size of this data has gone beyond human-sense capabilities and at this scale it’s almost impossible to detect patterns just by looking at the data. This is where data mining comes into the picture, it automates a part of this process to detect interpretable patterns.Deriving insights from data or data analytics can be classified into three forms: Descriptive, Predictive, and Prescriptive.If you look at data mining, it more or less fits into the first bracket i.e. descriptive. Data mining simplifies and summarizes the data making it easier for us to understand and derive our conclusions about specific cases/instances basis the patterns that data mining throws up. In other words, data mining describes the situation, explains what’s going on currently and helps you understand the situation in its entirety.How does it help businesses? For e.g. with data mining a company can identify their most profitable customers, offer that customer base better prices, also helping the company to accelerate its product innovation cycle. Data mining can help companies understand their current supply chains better and optimise it better.This was all about defining data mining, its use and importance. Now, coming to the data mining tools, you have a variety of techniques, including neural networks, and advanced statistics to locate patterns within the data and develop hypotheses. There are analytic tools like querying tools and today, live web-connectors and even analytics in the cloud which allows people, business and governments to be more nimble and agile, even as they conduct in-depth data mining.All of this does make data mining appear as this one really complicated field for which you need to have some special skills and should be ready to spend big monies to make sense of your data. With the rise of self-service analytics, there are tools or software available today provide you with many data mining workarounds depending upon the level and scale of data mining you expect. These self-service analytic tools not just help you understand your data better but also do it without leaving a hole in your pocket.(The author is country manager-India at Tableau Software)