「分かち書き」をしてみる

2022年9月3日

参考

インストール

「分かち書き」と「辞書」のライブラリをインストールする。

pip install mecab-python3
pip install unidic

unidic から辞書をダウンロードする。

python -m unidic download

コード

import MeCab

sample_txt = 'こんにちは、私はいまパイソンを勉強しています。'
wakati = MeCab.Tagger()
result = wakati.parse(sample_txt)

print(result)

実行結果は次の通りで「形態素解析」というのがされているようです。

こんにちは      感動
詞,一般,,,,,コンニチハ,今日は,こんにちは,コンニチワ,こんにちは,コンニチワ,混,"","","","","","",他,コンニチハ,コンニチハ,コンニチハ,コンニチハ,"5","","",37963946107478528,138112
、      補助記号,読点,,,,,,、,、,,、,,記号,"","","","","","",補助,,,,,"","","",6605693395456,24
私      代名詞,,,,,,ワタクシ,私-代名詞,私,ワタクシ,私,ワタクシ,和,"","","","","","",体,ワタクシ,ワタクシ,ワタクシ,ワタクシ,"0","","",11345327978324480,41274
は      助詞,係助詞,,,,,ハ,は,は,ワ,は,ワ,和,"","","","","","",係助,ハ,ハ,ハ,ハ,"","動詞%F2@0,名詞%F1,形容詞%F2@-1","",8059703733133824,29321
いま    名詞,普通名詞,副詞可能,,,,イマ,今,いま,イマ,いま,イマ,和,"","","","","","",体,イマ,イマ,イマ,イマ,"1","C3","",676208274711040,2460
パイソン        名詞,普通名詞,一般,,,,パイソン,パイソン-python,パイソン,パイソン,パイソン,パイソン,外,"","","","","","",体,パイソン,パイソン,パイソン,パイソン,"1","C1","",12090513360560640,43985
を      助詞,格助詞,,,,,ヲ,を,を,オ,を,オ,和,"","","","","","",格助,ヲ,ヲ,ヲ,ヲ,"","動詞%F2@0,名詞%F1,形容詞%F2@-1","",11381878116459008,41407
勉強    名詞,普通名詞,サ変可能,,,,ベンキョウ,勉強,勉強,ベンキョー,勉強,ベンキョー,漢,"","","","","","",体,ベンキョウ,ベンキョウ,ベンキョウ,ベンキョウ,"0","C2","",9415126692274688,34252
し      動詞,非自立可能,,,サ行変格,連用形-一般,スル,為る,し,シ,する,スル,和,"","","","","","",用,シ,スル,シ,スル,"0","C5","",5370298291593857,19537
て      助詞,接続助詞,,,,,テ,て,て,テ,て,テ,和,"","","","","","",接助,テ,テ,テ,テ,"","動詞%F1,形容詞%F2@-1","",6837321680953856,24874
い      動詞,非自立可能,,,上一段-ア行,連用形-一般,イル,居る,い,イ,いる,イル,和,"","","","","","",用,イ,イル,イ,イル,"0","C4","M4@1",710568013079169,2585
ます    助動詞,,,,助動詞-マス,終止形-一般,マス,ます,ます,マス,ます,マス,和,"","","","","","",助動,マス,マス,マス,マス,"","動詞%F4@1","",9812325267808939,35697
。      補助記号,句点,,,,,,。,。,,。,,記号,"","","","","","",補助,,,,,"","","",6880571302400,25
EOS

スペース区切りにする

インスタンス生成時に「-Owakati」を渡すようです。

wakati = MeCab.Tagger('-Owakati')

実行結果は次の通り。

こんにちは 、 私 は いま パイソン を 勉強 し て い ます 。

名詞のみ取得する

借り物のコードですが次の通り。

import collections
import MeCab

with open('sample.txt', 'r', encoding='UTF-8') as f:
    text = f.read()

# 読み込んだtextファイルで形態素解析を行う
tagger = MeCab.Tagger()
tagger.parse('')
node = tagger.parseToNode(text)

# 取り出す品詞を決めている.今回は名詞
word_list = []
while node:
    word_type = node.feature.split(',')[0]
    # 名詞の他にも動詞や形容詞なども追加できる
    if word_type in ["名詞"]:
        word_list.append(node.surface)
    node = node.next
word_chain = ' '.join(word_list)

# collections.counterでword_list内に含まれている名詞をカウント
c = collections.Counter(word_list)

# printでよく使われている単語top20を出力
print(c.most_common(20))

関連

2022年9月3日