Playing with Part of Speech (POS) taggers

Breaking text up into parts of speech is useful for a variety of tasks. When a query is sent to a search engine for example the part of speech of a word in the query will be a significant determinant of the results. 

Overall the aim of this project is to build a Named Entity Recognition system. But one of the useful feeds into this is creating a POS tagger. This will demonstrate some of the related techniques for working with language and the generated Parts of Speech will probably be useful features in the more advanced Named Entity Recognition model. 

So yesterday I said I would identify the data to use for training and testing and set up the shell of a github project.

Here is the project and the data I am using is from the Wall Street Journal. It was supplied as part of the Coursera course. However any POS tagged dataset should be fine. 

I tried 3 approaches

Find the most common tag for each word in the training set. Just use these (and a tag for Unknown) to make your predictions. A pretty basic method, but it gave about 59% accuracy, so a lot better than random guessing. 

Next I tried the same approach, but for bigrams. As expected this gives a much better result of about 70%. Still not great, but not too bad. 

Finally I tried a state of the art tagger which is part of the NLTK project. With minimal fuss this gives over 96% accuracy. Leagues ahead of my two efforts. 

So my next task is to build a sequence model with Keras to see if I can improve on my efforts here. The full code can be found in the repo above.

Comments

Post a Comment

Popular posts from this blog

Execute Jupyter notebooks line by line in VS Code

Normal notebook execution is cell by cell in Jupyter notebooks in the browser. This frequently results in cells needing to be split and rejoined at various points in the dev process. VS Code allows line by line execution of code within those cells. This feels a lot like running SQL code in MS SQL Analyzer and can be a very natural way to work. In addition VS Code gives you a nice view of the data returned from a cell or line execution with its Data Viewer. In WSL Ubuntu to get the line by line stuff working you will need to install the Jupyter Extension and the Python extension. Once these are in place the process of getting VS Code to find your correct python environment is very straightforward. You can find more about line by line debugging here: https://github.com/microsoft/vscode-jupyter/wiki/Setting-Up-Run-by-Line-and-Debugging-for-Notebooks Once the python extension is installed VS Code will find your virtual environments and you pyenv environments.
Read more

Using TensorFlow Serving

Having converted the front end to React the next step is to get a model making predictions from the server. TensorFlow Serving seems like the best way to do this. Instructions are here  for using Serving with Docker which is the recommended approach.  Some tips on docker: After you have run docker pull to get the TensorFlow Serving container you will want to list your containers: docker ps To run a container use docker run as per the instructions.  docker run -p 8501:8501 --mount type=bind,source=$(pwd)/data/model/pos_checkpoint,target=/models/pos_checkpoint -e MODEL_NAME=pos_checkpoint -t tensorflow/serving Then use docker stop XX where XX is enough of the tag from docker ps to identify that container.  You will need to put your SavedModel into an integer folder for the version.  The command to run a prediction using the REST endpoint is: curl -d '{"instances":[1,2,3]}' -X POST http://localhost:8501/v1/models/pos_checkpoint:predict That will error because the ...
Read more

Text Summarisation with BERT

Hugging Face provides a wide range of pretrained models for lots of machine learning tasks. Summarising the chapters of our novel - Bleak House - would be an interesting challenge. This could also be a very useful mnemonic technique if you were reviewing a long piece of work.  So here is the Hugging Face instructions on this. pip install transformers Then in python: from transformers import pipeline summarizer = pipeline("summarizer") Give it the text to summarize along with some params summarizer(ARTICLE, max_length=130, min_length=30, do_sample=False) BERT is a large model and takes about 15 minutes to download for me. However it is then a pip package and so will be available next time you try to run it. 
Read more