Key Diagnostic Metrics

Once we have the model we should evaluate its performance and tune it to improve its performance. If a data mining model is bad, it’s usually pretty easy to identify from the features, the validation procedure, or the goodness metrics. With diagnostic metrics, there are different measures, different metrics for different methods. First we will…

Text Mining – Sample Analysis using LightSide

Here is highlight of the text mining process in a simple text classification experiment  using LightSIDE. LightSide accepts the input data in the .csv format. The process in general has been outlined below: Data Preparation-> Features Extraction -> Model Generation Data preparation Data preparation is the first step in which the data is cleaned and…

Components to effective Predictive Modeling

Predictive modeling involves finding good subsets of predictors or explanatory variables. Models that fit the data well are better than models that fit the data poorly. Simple models are better than complex models. Working with a list of useful predictors, we can fit many models to the available data and then evaluate those models by…

Text Mining Overview

Text mining can be defined as the analysis of semi-structured or unstructured text data. The goal is to turn text information into numbers so that data  mining algorithms can be  applied. Text mining is an interdisciplinary field which incorporates data mining, web mining, information retrieval, information extraction, computational linguistics and natural language processing. Text mining…

Feature Engineering Introduction

When your goal is to get the best possible results from a predictive model, you need to get the most from what you have. This includes getting the best results from the algorithms you are using. It also involves getting the most out of the data for your algorithms to work with. How do you…

Predictive Modeling Overview

The raw data doesn’t offer a lot of value in its unprocessed state. However, by applying the right set of tools, we can pull powerful insights from it.With data in hand, you can begin doing analytics. Analytics can be used to make smart business decisions, compete and innovate. There are three types of data analysis:…