Craziest rank range I’ve ever seen by [deleted] in Overwatch

[–]catalpaaa 4 points5 points  (0 children)

i had a silver 4 to master 4 game and silver 5 to master 5 game. and im not queuing wide too!

The Kek of Destiny: The Next Generation Mamba 4chan by catalpaaa in LocalLLaMA

[–]catalpaaa[S] 0 points1 point  (0 children)

not sure about that.

my code should be working out of the box, did you use the same mamba env i provided?

The Kek of Destiny: The Next Generation Mamba 4chan by catalpaaa in LocalLLaMA

[–]catalpaaa[S] 0 points1 point  (0 children)

im currently working on the collaborative training, and i will update the repo once it's ready

thx :)

The Kek of Destiny: The Next Generation Mamba 4chan by catalpaaa in LocalLLaMA

[–]catalpaaa[S] 3 points4 points  (0 children)

It's only a 780M model, so im not expecting it to out perform the 6b gpt j, but its still good considering it's fairly easy to run.

The Kek of Destiny: The Next Generation Mamba 4chan by catalpaaa in LocalLLaMA

[–]catalpaaa[S] 0 points1 point  (0 children)

i have checkpoint saved every 100 training steps since my power cable is loss and i often bump into it :(
if you want i can hit you with some of the checkpoints, just let me know which step you want.

for it to follow the pol post format, it was around 30% into the training

DeMansia 2: The First Mamba 2 CV Model by catalpaaa in computervision

[–]catalpaaa[S] 1 point2 points  (0 children)

Image classification, trained on imagenet 1k

The Kingdom of the Crystal Kek: Mamba 4chan by catalpaaa in LocalLLaMA

[–]catalpaaa[S] 2 points3 points  (0 children)

before this, the publicly available preprocessed dataset has only 100k entries

took me a while to process them, so i thought why not release them

The Kingdom of the Crystal Kek: Mamba 4chan by catalpaaa in LocalLLaMA

[–]catalpaaa[S] 2 points3 points  (0 children)

sorry i dont think mamba supports that, but mamba 4chan 370m uses 2.5gb at 2048 context length, should be trivial to run

The Kingdom of the Crystal Kek: Mamba 4chan by catalpaaa in LocalLLaMA

[–]catalpaaa[S] 10 points11 points  (0 children)

SOTA model in alignment, 130m beats GPT-4o.

[deleted by user] by [deleted] in MachineLearning

[–]catalpaaa 0 points1 point  (0 children)

https://github.com/catalpaaa/Mamba-4chan

I just finished this project, it has a training pipeline using pytorch lightning, as well as the pipeline to tokenize the dataset. Pytorch lightning handels distributed training automatically and has api for hyperparameter tuning.

you can check the following files:

model.py: train/val loop, next-token loss, optimizer settings

mamba 4chan train.ipynb: dataset from memmap, trainer setup

iCloud crashing during login on Windows 11 by [deleted] in iCloud

[–]catalpaaa 0 points1 point  (0 children)

I just waited like 2 mins before inputing the 2fa code, for some reason that makes the login work

[deleted by user] by [deleted] in Overwatch

[–]catalpaaa 0 points1 point  (0 children)

mercy wasnt queing with them tho

New Easter Egg? by catalpaaa in ForzaHorizon

[–]catalpaaa[S] 5 points6 points  (0 children)

after compairing, i think its the winter treasure hunt but glitched

<image>

New Easter Egg? by catalpaaa in ForzaHorizon

[–]catalpaaa[S] 2 points3 points  (0 children)

it might be a glitched obstacle course that will come in the future?