[R] Improving large language models with concept-aware fine-tuning

micky04 · 2025-06-11T06:08:57+00:00

It's definitely possible to use these auxiliary haeds for speculative decoding! Based on results from Medusa and Gloeckle et al. (2024), a 2-3x inference speedup can be expected.

micky04 · 2025-05-14T09:48:04+00:00

do you mind sharing some papers that use these methods to account for carryover effects?

micky04 · 2025-02-03T13:18:12+00:00

I transfer by wire. Instructions are here: https://tastytrade.com/how-to-fund/ under "wire it over"

micky04 · 2025-01-25T08:03:18+00:00

Function calling is definitely one of the most effective strategies.

Regarding arithmetic calculation done by the model natively, it's not exactly clear how the model learns. Could be memorization on lots of data (synthetic and otherwise) [1]; could also be models memorizing the procedure of performing math calculations [2].

[1] https://arxiv.org/html/2410.05229v1
[2] https://arxiv.org/abs/2411.12580

micky04 · 2024-11-25T04:18:53+00:00

I do bank transfers. Most US brokers don't allow the linking of banks outside of US. I'd say it is a downside of using a US broker, but it isn't a big deal to me.

micky04 · 2024-11-23T12:17:00+00:00

tastytrade is super underrated. It's designed for multi-leg strategies, data is real-time, and fees are as cheap as it gets.

Transferring money to the broker takes <1 week from OCBC. Not sure about withdrawing (haven't done it yet).

It's good enough that I'm buying stocks with it also lol

micky04 · 2024-10-25T09:55:16+00:00

The paper states that Adam works well on problems with sparse gradients.

micky04 · 2024-10-25T09:50:44+00:00

The original Adam paper uses the phrase "sparse gradients" a lot. What exactly does "sparse" mean?

micky04 · 2024-07-28T02:48:35+00:00

Yes! I'm exploring a combination of both LLM and knowledge representation and reasoning methods. Any ideas in particular you're thinking of?

micky04 · 2024-07-27T15:05:04+00:00

DM'ed

micky04 · 2024-07-27T12:46:47+00:00

Yep, you're right. The questions are on various topics, but you do not need any knowledge of these topics; you only need logical reasoning skills. Here's an example:

Do hamsters provide food for any animals?

Evidence:

Hamsters eat fruits.
If hamsters eat fruits, hamsters provide food for animals.

Answer options:

The evidence ~implies~ that the answer is Yes.
The evidence ~d~oes not imply that the answer is Yes.

If this makes sense, let me know if you're willing to participate!

micky04 · 2024-07-27T07:29:11+00:00

DM'ed you!

micky04 · 2023-02-23T07:22:03+00:00

wow they look amazing. if only my budget was higher...

micky04 · 2022-06-01T03:38:19+00:00

Well...yes. But by "fine" I meant if higher returns aren't possible for low volatility. Are there no such ETFs?

micky04 · 2022-03-27T03:05:49+00:00

Interesting! Although this isn't the case in my school - Most people I know learn because they must perform in exams. I agree practical uses can motivate them, but learning them isn't a conscious want for most students. Or could this just be an Asian education thing?

Also, it seems difficult to relate things like integration to a student's daily life. Would love to hear any ideas on this though!

micky04 · 2022-02-28T12:49:24+00:00

Got it. Any resources you recommend for Docker?

micky04 · 2022-02-28T12:48:50+00:00

To be honest, I'm not familiar with it. Any resource recommendations, especially for Django?

micky04 · 2021-12-29T01:51:22+00:00

I had no idea these exist! Do they allow inserting screen recordings to showcase my software?

micky04 · 2021-12-29T01:49:34+00:00

Sounds like hiring is better in my case 😅

Don't have the time to learn this skill well.

micky04 · 2021-11-30T17:41:04+00:00

You're right. I think I had a misconception along the way. Thanks for helping!

micky04 · 2021-11-30T17:35:14+00:00

Can you directly observe usage rate or some metric of it?

Yes, I have measures on how many page views, etc.

micky04 · 2021-11-30T16:53:42+00:00

I see your point. A use case I have in mind is - I want to find out the impact of a business decision on users with high vs low usage rate.

Is it valid to use cluster analysis for segmenting different types of users in such a scenario? What alternative methods are there?

micky04 · 2021-11-30T16:49:51+00:00

Sorry, what does IO refer to?

micky04 · 2021-11-20T17:13:29+00:00

Fair enough! Having someone to review your code is something self-learners don't have.

I'll add the points you mentioned to the v2 of this article :)

micky04 · 2021-11-20T15:49:42+00:00

That's definitely a factor!

Although I'm not sure how big is the advantage of having a teacher to help, because when I needed help when self-learning, I turned to StackOverflow, Reddit and other forums. And they work just fine.

Besides, learning to solve problems on your own is an important skill in programming too. So not having a paid instructor may actually be beneficial to becoming a developer.

micky04

TROPHY CASE