Common NLP tasks such as named_entities, noun_chunks, etc.
9 |
|
16 | Return named-entities. |
17 |
|
19 | Return noun_chunks, group of consecutive words that belong together. |
20 |
Map words into vectors using different algorithms such as TF-IDF, word2vec or GloVe.
9 |
|
16 | Perform DBSCAN clustering. |
17 |
|
19 | Perform K-means clustering algorithm. |
20 |
|
22 | Perform mean shift clustering. |
23 |
|
25 | Perform non-negative matrix factorization. |
26 |
|
28 | Perform principal component analysis on the given Pandas Series. |
29 |
|
31 | Represent a text-based Pandas Series using term_frequency. |
32 |
|
34 | Represent a text-based Pandas Series using TF-IDF. |
35 |
|
37 | Perform TSNE on the given pandas series. |
38 |
Visualize insights and statistics of a text-based Pandas DataFrame.
9 |
|
16 | Show scatterplot using python plotly scatter. |
17 |
|
19 | Return a pandas series with index the top words and as value the count. |
20 |
|
22 | Plot wordcloud image using WordCloud from word_cloud package. |
23 |
named_entities(s, package='spacy')¶Return named-entities.
14 |Return a Pandas Series where each rows contains a list of tuples containing information regarding the given named entities.
15 |Tuple: (entity’name, entity’label, starting character, ending character)
16 |Under the hood, named_entities make use of Spacy name entity recognition.
17 |PERSON: People, including fictional.
NORP: Nationalities or religious or political groups.
FAC: Buildings, airports, highways, bridges, etc.
ORG : Companies, agencies, institutions, etc.
GPE: Countries, cities, states.
LOC: Non-GPE locations, mountain ranges, bodies of water.
PRODUCT: Objects, vehicles, foods, etc. (Not services.)
EVENT: Named hurricanes, battles, wars, sports events, etc.
WORK_OF_ART: Titles of books, songs, etc.
LAW: Named documents made into laws.
LANGUAGE: Any named language.
DATE: Absolute or relative dates or periods.
TIME: Times smaller than a day.
PERCENT: Percentage, including ”%“.
MONEY: Monetary values, including unit.
QUANTITY: Measurements, as of weight or distance.
ORDINAL: “first”, “second”, etc.
CARDINAL: Numerals that do not fall under another type.
Examples
41 |>>> import texthero as hero
42 | >>> import pandas as pd
43 | >>> s = pd.Series("Yesterday I was in NY with Bill de Blasio")
44 | >>> hero.named_entities(s)[0]
45 | [('Yesterday', 'DATE', 0, 9), ('NY', 'GPE', 19, 21), ('Bill de Blasio', 'PERSON', 27, 41)]
46 | clean(s: pandas.core.series.Series, pipeline=None) → pandas.core.series.Series¶Pre-process a text-based Pandas Series.
14 |texthero.preprocessing.fillna()
texthero.preprocessing.lowercase()
drop_no_content(s: pandas.core.series.Series)¶Drop all rows without content.
14 |Drop all rows from the given Pandas Series where has_content() is False.
Examples
16 |>>> s = pd.Series(["content", np.nan, "\t\n", " "])
17 | >>> drop_no_content(s)
18 | 0 content
19 | dtype: object
20 | get_default_pipeline() → List[Callable[[pandas.core.series.Series], pandas.core.series.Series]]¶Return a list contaning all the methods used in the default cleaning pipeline.
14 |texthero.preprocessing.fillna()
texthero.preprocessing.lowercase()
has_content(s: pandas.core.series.Series)¶Return a Boolean Pandas Series indicating if the rows has content.
14 |Examples
15 |>>> s = pd.Series(["content", np.nan, "\t\n", " "])
16 | >>> has_content(s)
17 | 0 True
18 | 1 False
19 | 2 False
20 | 3 False
21 | dtype: bool
22 | remove_angle_brackets(s: pandas.core.series.Series)¶Remove content within angle brackets <> and the angle brackets.
14 |See also
16 |remove_brackets()remove_round_brackets()remove_curly_brackets()remove_square_brackets()Examples
24 |>>> s = pd.Series("Texthero <is not a superhero!>")
25 | >>> remove_angle_brackets(s)
26 | 0 Texthero
27 | dtype: object
28 | remove_brackets(s: pandas.core.series.Series)¶Remove content within brackets and the brackets itself.
14 |Remove content from any kind of brackets, (), [], {}, <>.
15 |See also
17 |remove_round_brackets()remove_curly_brackets()remove_square_brackets()remove_angle_brackets()Examples
25 |>>> s = pd.Series("Texthero (round) [square] [curly] [angle]")
26 | >>> remove_brackets(s)
27 | 0 Texthero
28 | dtype: object
29 | remove_curly_brackets(s: pandas.core.series.Series)¶Remove content within curly brackets {} and the curly brackets.
14 |See also
16 |remove_brackets()remove_angle_brackets()remove_round_brackets()remove_square_brackets()Examples
24 |>>> s = pd.Series("Texthero {is not a superhero!}")
25 | >>> remove_curly_brackets(s)
26 | 0 Texthero
27 | dtype: object
28 | remove_diacritics(input: pandas.core.series.Series) → pandas.core.series.Series¶Remove all diacritics and accents.
14 |Remove all diacritics and accents from any word and characters from the given Pandas Series. Return a cleaned version of the Pandas Series.
15 |Examples
16 |>>> import texthero as hero
17 | >>> import pandas as pd
18 | >>> s = pd.Series("Noël means Christmas in French")
19 | >>> hero.remove_diacritics(s)
20 | 0 Noel means Christmas in French
21 | dtype: object
22 | remove_digits(input: pandas.core.series.Series, only_blocks=True) → pandas.core.series.Series¶Remove all digits and replace it with a single space.
14 |By default, only removes “blocks” of digits. For instance, 1234 falcon9 becomes ` falcon9`.
15 |When the arguments only_blocks is set to ´False´, remove any digits.
16 |See also replace_digits() to replace digits with another string.
Remove only blocks of digits.
22 |Examples
27 |>>> import texthero as hero
28 | >>> import pandas as pd
29 | >>> s = pd.Series("7ex7hero is fun 1111")
30 | >>> hero.preprocessing.remove_digits(s)
31 | 0 7ex7hero is fun
32 | dtype: object
33 | >>> hero.preprocessing.remove_digits(s, only_blocks=False)
34 | 0 ex hero is fun
35 | dtype: object
36 | Remove html tags from the given Pandas Series.
14 |Remove all html tags of the type <.*?> such as <html>, <p>, <div class=”hello”> and remove all html tags of type   and return a cleaned Pandas Series.
15 |Examples
16 |>>> s = pd.Series("<html><h1>Title</h1></html>")
17 | >>> remove_html_tags(s)
18 | 0 Title
19 | dtype: object
20 | remove_punctuation(input: pandas.core.series.Series) → pandas.core.series.Series¶Replace all punctuation with a single space (” “).
14 |remove_punctuation removes all punctuation from the given Pandas Series and replace it with a single space. It consider as punctuation characters all string.punctuation symbols !”#$%&’()*+,-./:;<=>?@[]^_`{|}~).
See also replace_punctuation() to replace punctuation with a custom symbol.
Examples
17 |>>> import texthero as hero
18 | >>> import pandas as pd
19 | >>> s = pd.Series("Finnaly.")
20 | >>> hero.remove_punctuation(s)
21 | 0 Finnaly
22 | dtype: object
23 | remove_round_brackets(s: pandas.core.series.Series)¶Remove content within parentheses () and parentheses.
14 |See also
16 |remove_brackets()remove_angle_brackets()remove_curly_brackets()remove_square_brackets()Examples
24 |>>> s = pd.Series("Texthero (is not a superhero!)")
25 | >>> remove_round_brackets(s)
26 | 0 Texthero
27 | dtype: object
28 | remove_square_brackets(s: pandas.core.series.Series)¶Remove content within square brackets [] and the square brackets.
14 |See also
16 |remove_brackets()remove_angle_brackets()remove_round_brackets()remove_curly_brackets()Examples
24 |>>> s = pd.Series("Texthero [is not a superhero!]")
25 | >>> remove_square_brackets(s)
26 | 0 Texthero
27 | dtype: object
28 | remove_stopwords(input: pandas.core.series.Series, stopwords: Union[Set[str], NoneType] = None, remove_str_numbers=False) → pandas.core.series.Series¶Remove all instances of words.
14 |By default uses NLTK’s english stopwords of 179 words:
15 |Set of stopwords string to remove. If not passed, by default it used NLTK English stopwords.
20 |Examples
25 |Using default NLTK list of stopwords:
26 |>>> import texthero as hero
27 | >>> import pandas as pd
28 | >>> s = pd.Series("Texthero is not only for the heroes")
29 | >>> hero.remove_stopwords(s)
30 | 0 Texthero heroes
31 | dtype: object
32 | Add custom words into the default list of stopwords:
35 |>>> import texthero as hero
36 | >>> from texthero import stopwords
37 | >>> import pandas as pd
38 | >>> default_stopwords = stopwords.DEFAULT
39 | >>> custom_stopwords = default_stopwords.union(set(["heroes"]))
40 | >>> s = pd.Series("Texthero is not only for the heroes")
41 | >>> hero.remove_stopwords(s, custom_stopwords)
42 | 0 Texthero
43 | dtype: object
44 | remove_urls(s: pandas.core.series.Series) → pandas.core.series.Series¶Remove all urls from a given Pandas Series.
14 |remove_urls remove any urls and replace it with a single empty space.
15 |See also
17 |texthero.preprocessing.replace_urls()Examples
22 |>>> import texthero as hero
23 | >>> import pandas as pd
24 | >>> s = pd.Series("Go to: https://example.com")
25 | >>> hero.remove_urls(s)
26 | 0 Go to:
27 | dtype: object
28 | remove_whitespace(input: pandas.core.series.Series) → pandas.core.series.Series¶Remove any extra white spaces.
14 |Remove any extra whitespace in the given Pandas Series. Removes also newline, tabs and any form of space.
15 |Useful when there is a need to visualize a Pandas Series and most cells have many newlines or other kind of space characters.
16 |Examples
17 |>>> import texthero as hero
18 | >>> import pandas as pd
19 | >>> s = pd.Series("Title \n Subtitle \t ...")
20 | >>> hero.remove_whitespace(s)
21 | 0 Title Subtitle ...
22 | dtype: object
23 | replace_punctuation(input: pandas.core.series.Series, symbol: str = ' ') → pandas.core.series.Series¶Replace all punctuation with a given symbol.
14 |replace_punctuation replace all punctuation from the given Pandas Series and replace it with a custom symbol. It consider as punctuation characters all string.punctuation symbols !”#$%&’()*+,-./:;<=>?@[]^_`{|}~).
Symbol to use as replacement for all string punctuation.
20 |Examples
25 |>>> import texthero as hero
26 | >>> import pandas as pd
27 | >>> s = pd.Series("Finnaly.")
28 | >>> hero.replace_punctuation(s, " <PUNCT> ")
29 | 0 Finnaly <PUNCT>
30 | dtype: object
31 | replace_stopwords(input: pandas.core.series.Series, symbol: str, stopwords: Union[Set[str], NoneType] = None) → pandas.core.series.Series¶Replace all instances of words with symbol.
14 |By default uses NLTK’s english stopwords of 179 words.
15 |Character(s) to replace words with.
20 |Set of stopwords string to remove. If not passed, by default it used NLTK English stopwords.
22 |Examples
27 |>>> s = pd.Series("the book of the jungle")
28 | >>> replace_stopwords(s, "X")
29 | 0 X book X X jungle
30 | dtype: object
31 | replace_urls(s: pandas.core.series.Series, symbol: str) → pandas.core.series.Series¶Replace all urls with the given symbol.
14 |replace_urls replace any urls from the given Pandas Series with the given symbol.
15 |See also
17 |texthero.preprocessing.remove_urls()Examples
22 |>>> import texthero as hero
23 | >>> import pandas as pd
24 | >>> s = pd.Series("Go to: https://example.com")
25 | >>> hero.replace_urls(s, "<URL>")
26 | 0 Go to: <URL>
27 | dtype: object
28 | stem(input: pandas.core.series.Series, stem='snowball', language='english') → pandas.core.series.Series¶Stem series using either porter or snowball NLTK stemmers.
14 |The act of stemming means removing the end of a words with an heuristic process. It’s useful in context where the meaning of the word is important rather than his derivation. Stemming is very efficient and adapt in case the given dataset is large.
15 |texthero.preprocessing.stem make use of two NLTK stemming algorithms known as nltk.stem.SnowballStemmer and nltk.stem.PorterStemmer. SnowballStemmer should be used when the Pandas Series contains non-English text has it has multilanguage support.
Stemming algorithm. It can be either ‘snowball’ or ‘porter’
21 |Supported languages: danish, dutch, english, finnish, french, german , hungarian, italian, norwegian, portuguese, romanian, russian, spanish and swedish.
23 |Notes
28 |By default NLTK stemming algorithms lowercase all text.
29 |Examples
30 |>>> import texthero as hero
31 | >>> import pandas as pd
32 | >>> s = pd.Series("I used to go \t\n running.")
33 | >>> hero.preprocessing.stem(s)
34 | 0 i use to go running.
35 | dtype: object
36 | tokenize(s: pandas.core.series.Series) → pandas.core.series.Series¶Tokenize each row of the given Series.
14 |Tokenize each row of the given Pandas Series and return a Pandas Series where each row contains a list of tokens.
15 |Algorithm: add a space between any punctuation symbol at 16 | exception if the symbol is between two alphanumeric character and split.
17 |Examples
18 |>>> import texthero as hero
19 | >>> import pandas as pd
20 | >>> s = pd.Series(["Today you're looking great!"])
21 | >>> hero.tokenize(s)
22 | 0 [Today, you're, looking, great, !]
23 | dtype: object
24 | kmeans(s: pandas.core.series.Series, n_clusters=5, init='k-means++', n_init=10, max_iter=300, tol=0.0001, precompute_distances='auto', verbose=0, random_state=None, copy_x=True, n_jobs=- 1, algorithm='auto')¶Perform K-means clustering algorithm.
14 |pca(s, n_components=2)¶Perform principal component analysis on the given Pandas Series.
14 |In general, pca should be called after the text has already been represented.
15 |Number of components to keep. If n_components is not set or None, all components are kept.
20 |Examples
25 |>>> import texthero as hero
26 | >>> import pandas as pd
27 | >>> s = pd.Series(["Sentence one", "Sentence two"])
28 | term_frequency(s: pandas.core.series.Series, max_features: Union[int, NoneType] = None, return_feature_names=False)¶Represent a text-based Pandas Series using term_frequency.
14 |Maximum number of features to keep.
19 |If True, return a tuple (term_frequency_series, features_names)
21 |Examples
26 |>>> import texthero as hero
27 | >>> import pandas as pd
28 | >>> s = pd.Series(["Sentence one", "Sentence two"])
29 | >>> hero.term_frequency(s)
30 | 0 [1, 1, 0]
31 | 1 [1, 0, 1]
32 | dtype: object
33 | To return the features_names:
36 |>>> import texthero as hero
37 | >>> import pandas as pd
38 | >>> s = pd.Series(["Sentence one", "Sentence two"])
39 | >>> hero.term_frequency(s, return_feature_names=True)
40 | (0 [1, 1, 0]
41 | 1 [1, 0, 1]
42 | dtype: object, ['Sentence', 'one', 'two'])
43 | tfidf(s: pandas.core.series.Series, max_features=None, min_df=1, return_feature_names=False)¶Represent a text-based Pandas Series using TF-IDF.
14 |Maximum number of features to keep.
19 |When building the vocabulary ignore terms that have a document frequency strictly lower than the given threshold.
21 |If True, return a tuple (tfidf_series, features_names)
23 |Examples
28 |>>> import texthero as hero
29 | >>> import pandas as pd
30 | >>> s = pd.Series(["Sentence one", "Sentence two"])
31 | >>> hero.tfidf(s)
32 | 0 [0.5797386715376657, 0.8148024746671689, 0.0]
33 | 1 [0.5797386715376657, 0.0, 0.8148024746671689]
34 | dtype: object
35 | To return the feature_names:
38 |>>> import texthero as hero
39 | >>> import pandas as pd
40 | >>> s = pd.Series(["Sentence one", "Sentence two"])
41 | >>> hero.tfidf(s, return_feature_names=True)
42 | (0 [0.5797386715376657, 0.8148024746671689, 0.0]
43 | 1 [0.5797386715376657, 0.0, 0.8148024746671689]
44 | dtype: object, ['Sentence', 'one', 'two'])
45 | tsne(s: pandas.core.series.Series, n_components=2, perplexity=30.0, early_exaggeration=12.0, learning_rate=200.0, n_iter=1000, n_iter_without_progress=300, min_grad_norm=1e-07, metric='euclidean', init='random', verbose=0, random_state=None, method='barnes_hut', angle=0.5, n_jobs=- 1)¶Perform TSNE on the given pandas series.
14 |Number of components to keep. If n_components is not set or None, all components are kept.
19 |scatterplot(df: pandas.core.frame.DataFrame, col: str, color: str = None, hover_data: ] = None, title='', return_figure=False)¶Show scatterplot using python plotly scatter.
14 |The name of the column of the DataFrame used for x and y axis.
19 |top_words(s: pandas.core.series.Series, normalize=False) → pandas.core.series.Series¶Return a pandas series with index the top words and as value the count.
14 |Tokenization: split by space and remove all punctuations that are not between characters.
15 |When set to true, return normalized values.
19 |wordcloud(s: pandas.core.series.Series, font_path: str = None, width: int = 400, height: int = 200, max_words=200, mask=None, contour_width=0, contour_color='PAPAYAWHIP', background_color='PAPAYAWHIP', relative_scaling='auto', colormap=None, return_figure=False)¶Plot wordcloud image using WordCloud from word_cloud package.
14 |Most of the arguments are very similar if not equal to the mother function. In constrast, all words are taken into account when computing the wordcloud, inclusive stopwords. They can be easily removed with preprocessing.remove_stopwords.
15 |Word are compute using generate_from_frequencies.
16 |Font path to the font that will be used (OTF or TTF). Defaults to DroidSansMono path on a Linux machine. If you are on another OS or don’t have this font, you need to adjust this path.
21 |Width of the canvas.
23 |Height of the canvas.
25 |The maximum number of words.
27 |When set, gives a binary mask on where to draw words. When set, width and height will be ignored and the shape of mask will be used instead. All white (#FF or #FFFFFF) entries will be considerd “masked out” while other entries will be free to draw on.
29 |If mask is not None and contour_width > 0, draw the mask contour.
31 |Mask contour color.
33 |Smallest font size to use. Will stop when there is no more room in this size.
35 |Background color for the word cloud image.
37 |Maximum font size for the largest word. If None, height of the image is used.
39 |Importance of relative word frequencies for font-size. With 41 | relative_scaling=0, only word-ranks are considered. With 42 | relative_scaling=1, a word that is twice as frequent will have twice 43 | the size. If you want to consider the word frequencies and not only 44 | their rank, relative_scaling around .5 often looks good. 45 | If ‘auto’ it will be set to 0.5 unless repeat is true, in which 46 | case it will be set to 0.
47 |Matplotlib colormap to randomly draw colors from for each word.
49 |This project is maintained by a dedicated group of people.
47 |This project is used by all these people
186 |This project is used by many folks
35 |Are you using this project?
38 | 39 | Add your company 40 | 41 |