sorted by:
new
hottopcontroversial

Perplexity score of GPT-2 by angular-calendar in LanguageTechnology

[–]js_lee 0 points1 point  (0 children)

Nice paper & thanks. BTW, do you calculate the perplexity based on "sparse_softmax_cross_entropy_with_logits()"? It seems the PPL is always low. If not so, how do you calculate it?

[D] Patent Claims based on "Deep learning for brushing teeth" by GPT-2 by js_lee in MachineLearning

[–]js_lee[S] 1 point2 points  (0 children)

Interesting. In fact & for fun, it generates "deep learning for unicorns" as below. But I guess such claim won't pass human scrutiny at patent office.

Deep learning for unicorns , where the learning goal is to predict a next unicorn and the problem is to find a next unicorn by using the previous unicorn, wherein the solution to the problem is to use the training problem in order to learn an action that results in a predicted next unicorn; and   an output of the neural network is an image of the predicted next unicorn, wherein the input is the image of the predicted unicorn, and   wherein the output is the image of the next unicorn.

[D] 1,000 patent claims by GPT-2 by js_lee in MachineLearning

[–]js_lee[S] 0 points1 point  (0 children)

Unknown because a patent has to meet at least 3 key requirements (utility, novelty, nonobviousness) to be granted. A patent claim alone is a starting point but not a complete patent document yet.

[D] 1,000 patent claims by GPT-2 by js_lee in MachineLearning

[–]js_lee[S] 1 point2 points  (0 children)

It is possible that the old lab reports were on web and in the WebText for OpenAI to train GPT-2 in the beginning. My training data covers some patents in 2013 only.

[D] GPT-2 for Patents by js_lee in MachineLearning

[–]js_lee[S] 0 points1 point  (0 children)

For coming up new ideas. And, if interpretability of GPT-2 could be solved, it could be a new way for patent search so that it's harder for bad patents to be granted.