Text Summarisation with BERT

Hugging Face provides a wide range of pretrained models for lots of machine learning tasks. Summarising the chapters of our novel - Bleak House - would be an interesting challenge. This could also be a very useful mnemonic technique if you were reviewing a long piece of work. 

So here is the Hugging Face instructions on this.

pip install transformers

Then in python:

from transformers import pipeline

summarizer = pipeline("summarizer")

Give it the text to summarize along with some params

summarizer(ARTICLE, max_length=130, min_length=30, do_sample=False)

BERT is a large model and takes about 15 minutes to download for me. However it is then a pip package and so will be available next time you try to run it. 


Comments

Post a Comment

Popular posts from this blog

Using curl scriptlets from a single file in VS Code

VSCode is a big step forward for code development. I found myself moving between consoles to run curl and the IDE to code in python on javascript. Easier if I could run lines of curl from a single bigger file of scripts. The way a notebook works.  Turns you can can do just that. Pull up the command list Cmd+shift+P. Locate 'run selected text in active terminal'. Click on the settings cog on the right hand side. Choose a shortcut key binding that does not clash with something else. VSCode will warn you.  And that is is. Saves a lot of hassle. Much easier to read and edit these than using the command line.
Read more

Recreating python feature extraction code in JavaScript

So TensorFlow's new Preprocessing Layers will make the use of models from python in TensorFlow.js much easier. At the moment those layers are only available in python, not JavaScript, so there is some transcription to be done.  The original model was trained by creating a windowed dataset with a sequence length of 5. In order to get prediction working in the browser that code needs to be replicated.  function createWindowedDataset ( data ){ let windowed = []; for ( let i = 0 ; i < data . length - contextSize ; i ++){ windowed [ i ]= data . slice ( i , i + contextSize ); } return windowed ; } Now my model loading function has changed to include a loop which loads a set of JSON files. There are 4 of these required to get the model predicting and to make sense of the predictions. So that code looks like this: const jsonToLoad = [ 'word_to_index.json' , 'pos_to_index.json' , 'most_common_tag_for_word.json' , 'index_to_...
Read more

Sentiment Analysis based on IMDB Movie Review dataset

Image
So the next step in analysing novels is to create a sentiment analyser. I got the IMDB movie review dataset from Kaggle here . There are a number of challenges here. The sentiment analysis I have done before was based on Twitter data. Short input data. Some of the inputs to the movie review db are over 1000 words long. So a histogram of the lengths of the reviews looks like this: A cutoff of 1000 max words seems sensible as a first pass. That still constitutes a lot of uninformative padding, but I can refine the method later.  So my first model looks like this: This gives categorical accuracy of about 84%. Not great, but could be worse.  I ran a bunch of predictions on movie reviews. Some that I made up and others from the test set. The model was 'correct' most of the time, but was very close to the middle of the distribution every time. It does not inspire much confidence.  So before I try to get this working on a long piece of work like a novel I need to improve the mod...
Read more