Skip to content Skip to sidebar Skip to footer

Python Nltk -- Stemming List Of Sentences/phrases

I have bunch of sentences in a list and I wanted to use nltk library to stem it. I am able to stem one sentence at a time, however I am having issues stemming sentences from a list

Solution 1:

You're passing a list to word_tokenize which you can't.

The solution is to wrap your logic in another for-loop,

data_list = ['the gamers playing games','higher scores','sports']
for words in data_list:
    words = tokenize.word_tokenize(words)
    for w in words:
        print(ps.stem(w))

>>>>the
gamer
play
game
higher
score
sport

Solution 2:

import nltk
from nltk.tokenize import sent_tokenize
from nltk.stem import PorterStemmer

sentence = """At eight o'clock on Thursday morning, Arthur didn't feel very good. So i take him to hospital."""

sentence = sentence.lower()

word_tokens = nltk.word_tokenize(sentence)
sent_tokens = sent_tokenize(sentence)

stemmer = PorterStemmer()
stemmed_word = []
stemmed_sent = []
for token in word_tokens:
    stemmed_word.append(stemmer.stem(token))
    
for sent_token in sent_tokens:
    stemmed_sent.append(stemmer.stem(sent_token))
    
print(stemmed_word)
print(stemmed_sent)

Post a Comment for "Python Nltk -- Stemming List Of Sentences/phrases"