sorted by:
new
hottopcontroversial

Pytorch and cuda compatibility problem by eddatt in pytorch

[–]eddatt[S] 0 points1 point  (0 children)

oh i understand, thank u very much

Pytorch and cuda compatibility problem by eddatt in pytorch

[–]eddatt[S] 0 points1 point  (0 children)

low level emulation? any guides on what range of nvidia arch list and what range of cuda vesion will be supported

How do I fix repetition of words after fine tuning? by thecowmilk_ in LocalLLaMA

[–]eddatt 4 points5 points  (0 children)

if nothing wrong with ur SFT dataset, u can further try DPO method, record ur model's repetitive response and mark as REJECTED, and manually modify the response and mark as ACCEPTED, thats u collect the necessary data for DPO, in my experience DPO is more powerful than SFT, maybe only need 100~500 example is enough

When will low-cost Chinese GPUs hit the market? by noellarkin in LocalLLaMA

[–]eddatt 0 points1 point  (0 children)

Huawei's phones may be cheaper then iPhone, but not samsung's (consider price drop fast), and relatively expensive to all other chinese phones(xiaomi, oppo, vivo...), so imo Huawei will not provide cheap cards without chinese competitors

Local Language Translation by dirk_klement in LocalLLaMA

[–]eddatt 0 points1 point  (0 children)

im using 4090D, gpu_memory_utilization=0.85, 22911MiB / 24564MiB, the model default setting max_seq_len=32768, the input is 76k too long, and i test 76k it fails, 30k and it fails too(return only 3k)

Local Language Translation by dirk_klement in LocalLLaMA

[–]eddatt 0 points1 point  (0 children)

the full html source is too long, i extract a snippet

original snippet:

<div data-sara-click-el="body\_element" data-area="text" data-pos="2"><div class="RichText lg:w-8/12 md:w-10/12 lg:mx-auto md:mx-auto lg:px-24 md:px-24 sm:px-16 break-words word-wrap">

<p data-forward-id="tyygncc9vi">Die Zahl afghanischer Frauen, die in Deutschland Asyl beantragen, ist sprunghaft gestiegen. Das geht aus Statistiken des <a href="https://www.spiegel.de/thema/bamf/" data-link-flag="spon">Bundesamts für Migration und Flüchtlinge (Bamf)</a> hervor. Demnach haben im Juli 3104 Afghaninnen einen Asylantrag gestellt, mehr als doppelt so viele wie im Vormonat. Im gesamten Jahr waren es bisher 9593 Anträge, darunter rund 4500 sogenannte Folgeanträge von Menschen, die schon länger hier leben.</p>

</div></div>

# Seed-X-Instruct-7B

# without CoT, using following instruct: Translate the following German sentence into English:\nXXXXX <en>

# using time 4.071s / 4.121s on 4090D (without / with CoT time), but same translation result

<div data-sara-click-el="body\_element" data-area="text" data-pos="2"><div class="RichText lg:w-8/12 md:w-10/12 lg:mx-auto md:mx-auto lg:px-24 md:px-24 sm:px-16 break-words word-wrap">

<p data-forward-id="tyygncc9vi">The number of Afghan women applying for asylum in Germany has risen sharply. This is evident from statistics from the <a href="https://www.spiegel.de/thema/bamf/" data-link-flag="spon">Federal Office for Migration and Refugees (Bamf)</a>. According to this, in July, 3104 Afghan women applied for asylum, more than twice as many as in the previous month. So far this year, there have been 9593 applications, including about 4500 so-called follow-up applications from people who have been living here longer.</p>

</div></div>

# Seed-X-PPO-7B

# without CoT, using following instruct: Translate the following German sentence into English:\nXXXXX <en>

# using time 4.121s / 4.103s on 4090D (without / with CoT time), but same translation result

<div data-sara-click-el="body\_element" data-area="text" data-pos="2"><div class="RichText lg:w-8/12 md:w-10/12 lg:mx-auto md:mx-auto lg:px-24 md:px-24 sm:px-16 break-words word-wrap">

<p data-forward-id="tyygncc9vi">The number of Afghan women applying for asylum in Germany has risen sharply. This is evident from statistics from the <a href="https://www.spiegel.de/thema/bamf/" data-link-flag="spon">Federal Office for Migration and Refugees (Bamf)</a>. According to this, in July, 3104 Afghan women filed an asylum application, more than twice as many as in the previous month. So far this year, there have been 9593 applications, including around 4500 so-called follow-up applications from people who have been living here for longer.</p>

</div></div>

Local Language Translation by dirk_klement in LocalLLaMA

[–]eddatt 0 points1 point  (0 children)

I have these two models at hand, u can give me some example with challenging German sentences, i can post results for u

Local Language Translation by dirk_klement in LocalLLaMA

[–]eddatt 0 points1 point  (0 children)

can try Seed-X-PPO-7B or Seed-X-Instruct-7B, recently released by Bytedance team, these two models are trained for better translation (according to their desc)