30 days of ML

The Named Entity Recognition is now working on the python side. So to recap: I trained a POS tagger to 90+% accuracy Used this as part of the feature set for a Named Entity Recognition model Tested this out on a novel - Bleak House So this is what the results look like. These are the most common named entities in the book. They match well with my knowledge from reading it.  Next up I want to get this working on the front end. Will tidy the code for the POS and NER up. Write the JavaScript version of the NER prediction code and get the novel loading code working. It would be interesting to see where these named entities feature throughout the book. A timeline of some sort would be good there. 

So the output from the tensorflow.js model works well and the POS tags that are being predicted match well those in the training text.  Next I am trying to run some predictions on a larger body of text. Extracting the Named Entities from a novel is one of the goals of this project. I have tested the predictions code on the python side, but just using the training data as input. This is fine for rough testing, but I need the python to be able to accept a large string of text and apply POS tags to every word.  I downloaded Bleak House from Project Gutenberg. Stripped out some of the pre and post boilerplate and tried to run some predictions on that.  I realised at this point that I had not made any allowance for unknown words. When you build your vocab from the training data and test with that too, this does not come up. I used defaultdicts to get this working.  The tags being predicted look wrong at this stage, but I will debug next. Once the POS tagger works on the p...

Quite a tricky thing to debug, but the full POS tagger is now working. You can find the code here . The challenge of encoding and decoding the data on the client was quite tricky. The JavaScript and layout is still very basic, but sufficient for now.  I have also improved the model to be about 96% accurate from the previous 92%. This involved adding a dense layer, dropout and training on more data. The WSJ dataset is very large, so I am still only using a fraction of this 50k out of 850k. I also removed a bunch of non POS tags. Things like punctuation. They may be useful for training some models, but mine just relied on words.  Next up is to train a Named Entity Recognition model using these Parts of Speech as part of the input features. 

So TensorFlow's new Preprocessing Layers will make the use of models from python in TensorFlow.js much easier. At the moment those layers are only available in python, not JavaScript, so there is some transcription to be done.  The original model was trained by creating a windowed dataset with a sequence length of 5. In order to get prediction working in the browser that code needs to be replicated.  function createWindowedDataset ( data ){ let windowed = []; for ( let i = 0 ; i < data . length - contextSize ; i ++){ windowed [ i ]= data . slice ( i , i + contextSize ); } return windowed ; } Now my model loading function has changed to include a loop which loads a set of JSON files. There are 4 of these required to get the model predicting and to make sense of the predictions. So that code looks like this: const jsonToLoad = [ 'word_to_index.json' , 'pos_to_index.json' , 'most_common_tag_for_word.json' , 'index_to_...