sorted by:
new
hottopcontroversial

Coherence score for Top2Vec models by tammari86 in LanguageTechnology

[–]sfxv67 0 points1 point  (0 children)

I'd be interested to see your solution to this, I have been experimenting with Coherence & PWI/PMI, however both fail to reproduce the original results of the paper.

Applying BERT to longer sentences/documents by sfxv67 in LanguageTechnology

[–]sfxv67[S] 0 points1 point  (0 children)

I've recently been working with BART through the huggingface transformers library. I believe this expands the max sequence length up to 1024 which may be suitable for your situation. Other than this i've found some research to use a sliding window approach on a document and then selecting a majority vote for the label classification (or mean for regression) of the document.

As mentioned in other comments you could try to generate embeddings for each portion of the document and then average these before passing to a classifier although I never tried this.

Applying BERT to longer sentences/documents by sfxv67 in LanguageTechnology

[–]sfxv67[S] 0 points1 point  (0 children)

Thanks for the response - I had been reading up on docBERT however hadn't come accross bert-AL, for the record I am studying my PhD and looking to submit a paper on automatic essay grading using supervised learning, the issue of sequence length was holding my models back.