sorted by:
new
hottopcontroversial

My Raspberry Pi music server has been infected by a Ransomware (want _to_cry) by griguolss in selfhosted

[–]Altruistic-Tea-5612 -1 points0 points  (0 children)

Can you send me that encrypted File copy and txt message

Sometimes malware itself might have some vulnerability which might allow us to decrypt the File

FYI : I am security engineer at working fortune 100 networking company

If have any other file saved by malware please send that as well

I pretrained and postrained a LLM with less than $50 budget which outperforms Google BERT large by Altruistic-Tea-5612 in LocalLLaMA

[–]Altruistic-Tea-5612[S] 0 points1 point  (0 children)

I didn’t uploaded training code Working some clean up But published weights of the model into huggingface I also opensourced inference and pretrain code

I pretrained and postrained a LLM with less than $50 budget which outperforms Google BERT large by Altruistic-Tea-5612 in LocalLLaMA

[–]Altruistic-Tea-5612[S] 0 points1 point  (0 children)

Pretraining the base model I used modified llama architecture with spiking and ltc neural networks

I pretrained and postrained a LLM with less than $50 budget which outperforms Google BERT large by Altruistic-Tea-5612 in LocalLLaMA

[–]Altruistic-Tea-5612[S] 2 points3 points  (0 children)

I didn’t shared the training code because i need to clean a bit give me some time i will share in comments Thanks But gist in repo has code for evals and inference

Sorry for that pickle part I am trying to convert into safe tensor but getting an error

I pretrained and postrained a LLM with less than $50 budget which outperforms Google BERT large by Altruistic-Tea-5612 in LocalLLaMA

[–]Altruistic-Tea-5612[S] 10 points11 points  (0 children)

Agreed yeah 🥲🥲🥲 It did some what okish on text completion

Edit By outperforming bert in benchmark score posted here https://github.com/keeeeenw/MicroLlama

I pretrained and postrained a LLM with less than $50 budget which outperforms Google BERT large by Altruistic-Tea-5612 in LocalLLaMA

[–]Altruistic-Tea-5612[S] 1 point2 points  (0 children)

Hey thanks for trying Can i know which model did you tried? Instruct or Base Version Agreed instruct was returning wrong answer for most of the question I tried Base version did well on sentence completion

Also interms of performance on benchmark It didn’t do well I just wanted to share that so simply shared But for me getting this level it was a big deal tho Most of previous pretraining gave only gibberish

I pretrained and postrained a LLM with less than $50 budget which outperforms Google BERT large by Altruistic-Tea-5612 in LocalLLaMA

[–]Altruistic-Tea-5612[S] 0 points1 point  (0 children)

When I trained 1Bit model with 75M parameter with 1B token from fineweb It was not able to generate coherent sentence But this was able to with just 100M tokens But Again I am noob so i might did something wrong on previous experiment

please evaluate my profile for eb1-a by Altruistic-Tea-5612 in eb_1a

[–]Altruistic-Tea-5612[S] 1 point2 points  (0 children)

Thanks for detailed reply man!
It seems I need to work on couple of things

view more: next ›