aug.augment(Orignial_text) gives error

[ashwanik94]

Some weights of GPT2LMHeadModel were not initialized from the model checkpoint at distilgpt2 and are newly initialized: ['transformer.h.0.attn.masked_bias', 'transformer.h.1.attn.masked_bias', 'transformer.h.2.attn.masked_bias', 'transformer.h.3.attn.masked_bias', 'transformer.h.4.attn.masked_bias', 'transformer.h.5.attn.masked_bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

TypeError Traceback (most recent call last)
in ()
1
2 aug = nas.ContextualWordEmbsForSentenceAug(model_path='distilgpt2')
----> 3 augmented_text = aug.augment(Orignial_text)
4 print("Original:")
5 print(Orignial_text)

5 frames
/usr/local/lib/python3.6/dist-packages/nlpaug/base_augmenter.py in augment(self, data, n, num_thread)
84 # TODO: support multiprocessing for GPU
85 # https://discuss.pytorch.org/t/using-cuda-multiprocessing-with-single-gpu/7300
---> 86 augmented_results = [action_fx(clean_data) for _ in range(n)]
87 else:
88 augmented_results = self._parallel_augment(action_fx, clean_data, n=n, num_thread=num_thread)

/usr/local/lib/python3.6/dist-packages/nlpaug/base_augmenter.py in (.0)
84 # TODO: support multiprocessing for GPU
85 # https://discuss.pytorch.org/t/using-cuda-multiprocessing-with-single-gpu/7300
---> 86 augmented_results = [action_fx(clean_data) for _ in range(n)]
87 else:
88 augmented_results = self._parallel_augment(action_fx, clean_data, n=n, num_thread=num_thread)

/usr/local/lib/python3.6/dist-packages/nlpaug/augmenter/sentence/context_word_embs_sentence.py in insert(self, data)
112 text += ' ' + self.model.MASK_TOKEN
113
--> 114 outputs = self.model.predict(text, n=1, external_memory=external_memory)
115 results = outputs[0]
116 if results is None:

/usr/local/lib/python3.6/dist-packages/nlpaug/model/lang_models/gpt2.py in predict(self, text, target_word, n, external_memory)
46 target_token_logits, target_token_idxes = self.filtering(target_token_logits, seed)
47 if len(target_token_idxes) != 0:
---> 48 results = self.pick(target_token_logits, target_token_idxes, target_word=target_word, n=n)
49 else:
50 results = None

/usr/local/lib/python3.6/dist-packages/nlpaug/model/lang_models/language_models.py in pick(self, logits, idxes, target_word, n)
87
88 def pick(self, logits, idxes, target_word, n=1):
---> 89 candidate_ids, candidate_probas = self.prob_multinomial(logits, n=n*10)
90 candidate_ids = [idxes[candidate_id] for candidate_id in candidate_ids]
91 results = self.get_candidiates(candidate_ids, candidate_probas, target_word, n)

/usr/local/lib/python3.6/dist-packages/nlpaug/model/lang_models/language_models.py in prob_multinomial(self, logits, n)
101
102 # Draw candidates
--> 103 num_sample = min(n, torch.nonzero(probas).size(0), as_tuple=False) # Number of potential candidate is small when top_k/ top_p are used.
104 filtered_top_n_ids = torch.multinomial(probas, num_samples=num_sample, replacement=False).tolist()
105 # filtered_top_n_ids = np.random.choice(probas.size(0), num_sample, False, probas.cpu().numpy()).tolist()

TypeError: 'as_tuple' is an invalid keyword argument for this function

[makcedward]

Try to upgrade transformer library version to >=3.0.0 and torch >= 1.6.0 . "as_tuple" paramter is introduced from 0.0.15 version

[ashwanik94]

Python Version 3.6.9
Tensorflow version 2.3.0
Torch Version 1.6.0+cu101
transformers version 3.0.2
nlpaug version 0.0.15

Still getting the same error with this code:

import nlpaug.augmenter.word as naw
from nlpaug.util import Action
from transformers import XLNetTokenizer, XLNetModel, GPT2Model, GPT2PreTrainedModel, GPT2Tokenizer, BertTokenizer

text = 'The quick brown fox jumps over the lazy dog .'

aug = naw.ContextualWordEmbsAug()
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented Text:")
print(augmented_text)

error: 'as_tuple' is an invalid keyword argument for this function

[makcedward]

Fixed the bug. You may upgrade to 0.0.16 version

[ashwanik94]

Thanks for the quick fix. It's working fine now with 0.0.16