What will I do for the next 30 days?

I have been taking online courses in ML for several years now, mainly on Coursera. I do some ML in work too, but want to use this space to push my skills further. I have a lot of ideas of things to do with ML, but usually fail to deliver a finished product. There are a bunch of reasons, but distraction and busyness are high on this list. 

So I will start simple and try to build a straightforward model to make some predictions. Ideally that will go into a web page so that the end result can be demonstrated. 

I have just completed this course in Natural Language Processing. Lots of interest in there, but one of the simpler models was for Named Entity Recognition. Given a body of text - a book say - can a model find out what are the named entities - people and places for example?

This type of model forms the basis for a bunch of other useful functions. How often does each character appear in the book? Using sentiment analysis across the span of the book is it possible to determine which characters are associated with positive sentiment and which negative?

Anyway, NER first, other ideas maybe later. 

This project is limited to spare time for 30 days. I have learned a lot over the last few years, so should be well able to come up with something useful in that time. 


So today I will identify a set of data to use and set up the project in git.

Comments

Post a Comment

Popular posts from this blog

Recreating python feature extraction code in JavaScript

So TensorFlow's new Preprocessing Layers will make the use of models from python in TensorFlow.js much easier. At the moment those layers are only available in python, not JavaScript, so there is some transcription to be done.  The original model was trained by creating a windowed dataset with a sequence length of 5. In order to get prediction working in the browser that code needs to be replicated.  function createWindowedDataset ( data ){ let windowed = []; for ( let i = 0 ; i < data . length - contextSize ; i ++){ windowed [ i ]= data . slice ( i , i + contextSize ); } return windowed ; } Now my model loading function has changed to include a loop which loads a set of JSON files. There are 4 of these required to get the model predicting and to make sense of the predictions. So that code looks like this: const jsonToLoad = [ 'word_to_index.json' , 'pos_to_index.json' , 'most_common_tag_for_word.json' , 'index_to_...
Read more

Named Entity Recognition for Bleak House

Image
The Named Entity Recognition is now working on the python side. So to recap: I trained a POS tagger to 90+% accuracy Used this as part of the feature set for a Named Entity Recognition model Tested this out on a novel - Bleak House So this is what the results look like. These are the most common named entities in the book. They match well with my knowledge from reading it.  Next up I want to get this working on the front end. Will tidy the code for the POS and NER up. Write the JavaScript version of the NER prediction code and get the novel loading code working. It would be interesting to see where these named entities feature throughout the book. A timeline of some sort would be good there. 
Read more

Using curl scriptlets from a single file in VS Code

VSCode is a big step forward for code development. I found myself moving between consoles to run curl and the IDE to code in python on javascript. Easier if I could run lines of curl from a single bigger file of scripts. The way a notebook works.  Turns you can can do just that. Pull up the command list Cmd+shift+P. Locate 'run selected text in active terminal'. Click on the settings cog on the right hand side. Choose a shortcut key binding that does not clash with something else. VSCode will warn you.  And that is is. Saves a lot of hassle. Much easier to read and edit these than using the command line.
Read more