Stokastik

Machine Learning, AI and Programming

Category: PROGRAMMING

Building a Neural Network from scratch in Python

In this post I am going to build an artificial neural network from scratch. Although there exists lots of advanced libraries for neural networks written using a variety of programming languages, the idea is not to re-invent the wheel but to understand what are the components required to make a workable neural network. Although a full-fledged industrial scale neural network might require a lot of research and experimentations with the datasets. Building a simple […]

Continue Reading →

Building an Incremental Named Entity Recognizer System

In the last post, we saw how to train a system to identify Part Of Speech tags for words in sentences. In essence we found out that discriminative models such as Neural Networks and Conditional Random Fields, outperforms other methods by 5-6% in prediction accuracy. In this post, we will look at another common problem in Natural Language Processing, known as the Named Entity Recognition (NER in short). The problem […]

Continue Reading →

Building a POS Tagger with Python NLTK and Scikit-Learn

In this post we are going to understand about Part-Of-Speech Taggers for the English Language and look at multiple methods of building a POS Tagger with the help of the Python NLTK and scikit-learn libraries. The available methods ranges from simple regular expression based taggers to classifier based (Naive Bayes, Neural Networks and Decision Trees) and then sequence model based (Hidden Markov Model, Maximum Entropy Markov Model and Conditional Random […]

Continue Reading →

Generative vs. Discriminative Spell Corrector

We have earlier seen two approaches of doing spelling corrections in text documents. Most of the spelling errors encountered are in either user generated contents or OCR outputs of document images. Presence of spelling errors introduce noise in data and as a result impact of important features gets diluted. Although the methods explained are  different in how they are implemented but theoretically both of them work on the same principle. […]

Continue Reading →

Using Word Vectors in Multi-Class Text Classification

Earlier we have seen how instead of representing words in a text document as isolated features (or as N-grams), we can encode them into multidimensional vectors where each dimension of the vector represents some kind semantic or relational similarity with other words in the corpus. Machine Learning problems such as classification or clustering, requires documents to be represented as a document-feature matrix (with TF or TF-IDF weighting), thus we need some […]

Continue Reading →

Designing a Contextual Graphical Model for Words

I have been reading about Word Embedding methods that encode words found in text documents into multi-dimensional vectors. The purpose of encoding into vectors is to give "meaning" to words or phrases in a context. Traditional methods of document classification treat each word in isolation or at-most use a N-gram approach i.e. in vector space, the words are represented as one-hot vectors which are sparse and do not convey any meaning whereas […]

Continue Reading →

Understanding Word Vectors and Word2Vec

Quite recently I have been exploring the Word2Vec tool, for representing words in text documents as vectors. I got the initial ideas about word2vec utility from Google's code archive webpage. The idea behind coming up with this kind of utility caught my interest and later I went on to read the following papers by Mikolov et. al. to better understand the algorithm and its implementation. Efficient Estimation of Word Representations […]

Continue Reading →

Learning From Unlabelled Data - EM Approach

Accurately labelled data can be a bottleneck in many machine learning problems as they are difficult and expensive to obtain, and even if we obtain some labelled data, the labels might not be 100% accurate. Many startups working in machine learning space resort to crowdsourcing of the labelling task. Inspired by this research paper, I am going to try and use lots of unlabelled data in addition to small amounts of labelled data […]

Continue Reading →

Building a Classification Tree from scratch

In this post I am going to demonstrate an implementation of a classification tree from scratch for multi-label classification. Since most of my work involves working with text classification, hence the classification tree that I am going to demonstrate here has been built, keeping text classification problems in mind. The theoretical explanation about various components and modules about classification tree can be found in this paper. This implementation is not ready-made for […]

Continue Reading →

Logistic Regression Analysis with Examples using R

In the last post we had seen how to perform a linear regression on a dataset with R. We had also seen how to interpret the outcome of the linear regression model and also analyze the solution using the R-Squared test for goodness of fit of the model, the t-test for significance of each variable in the model, F-statistic for significance of the overall model, Confidence intervals for the variable […]

Continue Reading →