Data Mining

This tab contains information and download links for two books related to data mining:

1. Data Mining Algorithms in C++
2. Extracting and Selecting Features for Data Mining

NOTE on platforms... All of my code was developed using Microsoft Visual C++ 2010 and 2019 and is geared toward a Windows environment. If you will use it on a different platform such as Linux, you will need to make some modifications to the header files, function declarations, and linked routines.

Data Mining Algorithms in C++

Data mining is a broad, deep, and frequently ambiguous field. I've been blessed to work as a consultant in a wide variety of fields, enjoying rare diversity in my work. Early in my career, I developed computer algorithms that examined high-altitude photographs in an attempt to discover useful things. Eventually I moved on to the medical field, and then finance. All of these endeavors have something in common: they all require that we find variables that are meaningful in the context of the application.

I have taken some of my most-used techniques, those that I have found to be especially valuable in the study of relationships among variables, and documented them with basic theoretical foundations and well commented C++ source code. Some special topics not frequently covered in other sources:

• Information theory as a foundation for discovering relationships between variables
• Uncertainty reduction using information theory
• Schreiber's information transfer — directional transfer of information from one time series to another
• Forward stepwise selection generalized to ranked sets of predictor candidates
• Simple modifications to relationship criteria to detect profoundly nonlinear relationships
• Monte-Carlo permutation tests for rigorous statistical relationship tests
• Combinatorially Symmetric Cross Validation for detecting overfitting
• Automated selection of variables suited for predicting a given target, including target selection
• Feature Weighting as Regularized Energy-Based Learning (FREL)
• Generalized scatterplots highlighting relationship anomalies
• Maximum likelihood factor analysis and varimax rotation
• Clustering variables within confines of a particular subspace of interest
• Detecting when time-series variables move between coherent and independent states

If you would like to download the Table of Contents, click here.

To download a zip file containing all of the source code referenced in the book, click here.

To download the user's manual for the DATAMINE program, click here.

To download a zip file containing the DATAMINE program and its manual, click here.

Extracting and Selecting Features for Data Mining

My book "Extracting and Selecting Features for Data Mining: Algorithms in C++ and CUDA C" has been licensed by the Apress division of Springer. The following topics are covered:

• Hidden Markov models chosen and optimized according to their multivariate correlation with a target
• Forward Selection Component Analysis
• Local Feature Selection — identifies predictors optimal in localized areas of the feature space
• Stepwise selection of predictive features enhanced in three important ways
• Nominal-to-ordinal conversion

To download a zip file containing all of the source code referenced in the book, click here.

NOTE on downloading Zip files: As of this writing, GoDaddy has a bug in their website support which often prevents them from automatically detecting Zip files correctly. They will tell you it is a file of unknown type with a long name. You will need to download this file and then manually add a .ZIP extension, which will allow you to unzip the file. I apologize for this, but there is nothing I can do about it.