Python - Chunks and Chinks

La segmentation est le processus de regroupement de mots similaires en fonction de la nature du mot. Dans l'exemple ci-dessous, nous définissons une grammaire par laquelle le bloc doit être généré. La grammaire suggère la séquence des phrases comme les noms et les adjectifs, etc. qui sera suivie lors de la création des blocs. La sortie illustrée des morceaux est indiquée ci-dessous.

import nltk
sentence = [("The", "DT"), ("small", "JJ"), ("red", "JJ"),("flower", "NN"), 
("flew", "VBD"), ("through", "IN"),  ("the", "DT"), ("window", "NN")]
grammar = "NP: {
      
       ?
       

        *
        
         }" cp = nltk.RegexpParser(grammar) result = cp.parse(sentence) print(result) result.draw()

When we run the above program we get the following output −

chunk_1.PNG

Changing the grammar, we get a different output as shown below. 

import nltk
sentence = [("The", "DT"), ("small", "JJ"), ("red", "JJ"),("flower", "NN"),
 ("flew", "VBD"), ("through", "IN"),  ("the", "DT"), ("window", "NN")]
grammar = "NP: {
      

       ?
       
        *
        
         }" chunkprofile = nltk.RegexpParser(grammar) result = chunkprofile.parse(sentence) print(result) result.draw()

When we run the above program we get the following output −

chunk_2.PNG

Chinking

Chinking is the process of removing a sequence of tokens from a chunk. If the sequence of tokens appears in the middle of the chunk, these tokens are removed, leaving two chunks where they were already present. 

import nltk
sentence = [("The", "DT"), ("small", "JJ"), ("red", "JJ"),("flower", "NN"), ("flew", "VBD"), ("through", "IN"),  ("the", "DT"), ("window", "NN")]
grammar = r"""
  NP:
    {<.*>+}         # Chunk everything
    }
      
       +{      # Chink sequences of JJ and NN
  """
chunkprofile = nltk.RegexpParser(grammar)
result = chunkprofile.parse(sentence) 
print(result)
result.draw()

When we run the above program, we get the following output −

chink.PNG

As you can see the parts meeting the criteria in grammar are left out from the Noun phrases as separate chunks. This process of extracting text not in the required chunk is called chinking.