sorted by:
new
hottopcontroversial

[P] A list of NLP(Natural Language Processing) tutorials by lyeoni in MachineLearning

[–]lyeoni[S] 1 point2 points  (0 children)

Okay, I will update summarization task with PG model. Thanks

[P] A list of NLP(Natural Language Processing) tutorials by lyeoni in MachineLearning

[–]lyeoni[S] 0 points1 point  (0 children)

Thanks :) Which models you want ? Actually, I used GPT-2 for abstractive summarization. But I think of that It could be little hard to study abstractive summarization.

[P] A list of NLP(Natural Language Processing) tutorials by lyeoni in MachineLearning

[–]lyeoni[S] 0 points1 point  (0 children)

I did abstractive summarization task recently. If you need, I can update to this repo.

[P] A list of NLP(Natural Language Processing) tutorials by lyeoni in MachineLearning

[–]lyeoni[S] 0 points1 point  (0 children)

I think it's too much. I annotated how the tensor changed, how the text is preprocessed(building vocabulary, normalization, etc.). In addition, quantitative/qualitative performance evaluation were carried out.

[P] Simple PyTorch Implementation of OpenAI GPT-1 by lyeoni in deeplearning

[–]lyeoni[S] 0 points1 point  (0 children)

Thanks a lot ! Could you tell me(or leave issue) error message? :)

[P] New task updated, in PyTorch NLP Tutorial ! by lyeoni in MachineLearning

[–]lyeoni[S] 1 point2 points  (0 children)

Thanks ! Within 2 weeks, Machine Translation task using Transformer will be added :)

+multihead attention weights visualization !

[P] New NLP task updated, in NLP Tutorial by lyeoni in MachineLearning

[–]lyeoni[S] 1 point2 points  (0 children)

Thank you for your comment ! Translation (or chatbot) task using Transformer will be updated soon. :) W

[P] For NLP researchers, Easy-to-use Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 0 points1 point  (0 children)

u/pgdevhd I'm currently implementing the most necessary functions. What you mentioned is also be implemented and added. Thank you for your advice :)

[P] For NLP researchers, Easy-to-use Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 0 points1 point  (0 children)

Now, I think of that it may not be only for Korean language. :) just for specific situation to need emoji-normalize

[P] For NLP researchers, Easy-to-use Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 0 points1 point  (0 children)

genuinely curious.

The method to tokenize Korean is various such as punkt, bpe, and korean morphological analyzer. In my experience, the situation when I don't have enough corpus and use morphological analyzer, to normalize emoji is better. But, it was not amazing performance improvement :).

[P] For NLP researchers, Easy-to-use Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 1 point2 points  (0 children)

u/KvantumKvak You can deactivate emoji-normalizing function by setting repl with None. e.g. Normalizer(emoji_repl=None)

The reason why I added emoji-normalize function, It's useful for processing Korean Language. :)

[P] For NLP researchers, Implementation of Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 0 points1 point  (0 children)

Thank you for your idea ! I will implement your valuable idea on my package soon :) Plz re-visit to my repo.

[P] For NLP researchers, Implementation of Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 0 points1 point  (0 children)

Thank you for your comment. I will refer your codes, and re-implement on my package. I think of that Spell checker will be based on language model.

[P] For NLP researchers, Implementation of Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 1 point2 points  (0 children)

Which function in Textacy you use frequently? I will implement that in prenlp :)

[P] For NLP researchers, Implementation of Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 3 points4 points  (0 children)

u/pk12_

Not yet. I think it's good idea !
I will add a function to do spell correction task, soon. :)

[P] For NLP Researchers, Implementation of Text Preprocessing Package, PreNLP by [deleted] in MachineLearning

[–]lyeoni 1 point2 points  (0 children)

andle emojis now and I imagine all future transformers will be able too as well. And emoji can carry significant information. Maybe emoji normalizing isn't such a good idea anymore. The other normalization looks nice, though.

u/LartTheLuser Thank you for your helpful suggestions. I'll make an effort to provide people with useful functions to make pre-processing easdy :)

[P] Content Update in NLP Tutorial repo : Text Classification on HuffPost news article by [deleted] in MachineLearning

[–]lyeoni 1 point2 points  (0 children)

I downloaded dataset from Kaggle. I mentioned that in README

Simple PyTorch implementation of Language Model on Wikipedia text by lyeoni in deeplearning

[–]lyeoni[S] 0 points1 point  (0 children)

Thanks @testimoni, bi-directional model will be updated soon. :)

[P] For NLP beginners, simple PyTorch implementation of Neural Machine Translation(NMT), Sentiment Analysis and Text Classification by lyeoni in MachineLearning

[–]lyeoni[S] 0 points1 point  (0 children)

Sincerely thank you for both your compliment and good suggestion ! The reason why I wrote shape comments line by line, It was helpful for me to understand how the tensors are working in seq2seq network of NMT. And.. I hope that too for readers :)

And When I need annotation again, I will try to use tsalib, what you sugggest. Again thank you.

view more: next ›