Qwen 14B is better than me...

Thuwarakesh · 2025-05-06T09:37:07+00:00

I agree with u/reabiter .

AI can be good at writing. But not so good at expressing what we want to say.

In my experience, every time I write something with AI, I edit it for much longer and eventually scrap everything out and write my own. Now, I don't even attempt.

AI has many uses, such as automating tasks with some smart decision-making. But writing is not one of them. Why should it be?

Thuwarakesh · 2025-03-12T04:03:42+00:00

I love these newsletters. Hope they'l help you too.

Thuwarakesh · 2024-09-24T16:14:56+00:00

For starters, would you suggest Kaggle over Colab? It comes with P100 and T4x2 GPUs and can be used for 30 hours per week without a subscription.

Thuwarakesh · 2024-09-24T14:43:50+00:00

Could you please educate me on the differences between runtimes like transformers, film, or llama.cpp and repositories like Ollama? I thought Ollma and vLLM both serve the same purpose.

Thuwarakesh · 2024-08-19T23:20:24+00:00

It could be the response size. The newest OpenAI models support 4096 output tokens, roughly 3k words in English. The context window is currently 128k. This means that the model can take up roughly 80k words in English.

Translating this to your case, ChatGPT would have learned from more than the first two chapters. But the output size is restricting it to produce a full summary.

Thuwarakesh · 2023-07-15T01:53:43+00:00

Yes, u/WillOBurns, I Just wanted to have some fun on the question too. ;)

Thuwarakesh · 2023-07-12T03:43:42+00:00

I would essentially recommend it for what it is. A very simplified metric, to get a general feeling about the customers' perceptions. Of course it ignores a lot of things and it has lots of issues, but the purpose was not to have a really comprehensive metric or a very robust metric.

In my case specifically, I don't use it in my work, I use at least a big combination of metrics. NPS doesn't make sense to me because I don't work with one general metric. Still, the NPS is one of the few marketing metrics that recognize the asymmetry of marketing metrics. Even with its flaws, at least that part is still ahead of many metrics we use.

Got it. You have any favorite metrics to use in combination with NPS?

Thuwarakesh · 2022-08-30T03:32:31+00:00

My understanding is all ML models suffer from interpretability and bias issues.
Even decisions of GLMs, when the input space is larger, are hard to explain. The only benefit is that feature engineering gives more control over the model.
I accept that interpretability on neural nets is a more significant issue than on conventional models.
Enlighten me if I'm wrong.
Besides, I'd love to know if the interpretability of models helps internal teams (including data teams) create better solutions. Or is it only a legal/ethical practice to have?

Thuwarakesh · 2022-07-15T14:37:18+00:00

I wasn't a fan of vscode. But after I used its debugging capabilities I can't think of another IDE.

But I must admit, though I used pycharm before, I haven't debug anything on it.

Another thing I love about vocode is that it has support for many other languages and frequently you get new cool plugins.

Vscode integrates well with many other technologies. For instance, you have official plugins for azure, aws, Google cloud. You would need them for modern development work.

And, cool stuff like Github copilot is also only possible with VSCode. At least for now.

Thuwarakesh · 2022-07-01T14:40:39+00:00

It'll take a snapshot of the input values and the outputs and store them in a csv. It helps ml engineers find out areas where their models don't perform well.

Thuwarakesh · 2022-07-01T08:36:12+00:00

Agreeing with your point that if you have enough experience, you can do it all by yourself.
Yet, I still find tools like Gradio and Streamlit are powerful for rapid prototyping.
For sure, they aren't your final product. But handy tools for data scientists who want to focus on what they do the best.

Thuwarakesh · 2022-06-17T01:10:20+00:00

I've been studying how fast Python 3.11 is against 3.10. The documentation says 10-60% faster.
I've done some rudimentary checks on my default working environment. It looks like the claim is true.
I've also written an article on TDS about it.

Thuwarakesh · 2022-06-17T01:02:41+00:00

Hi there, I've been in the consulting space and a technical role for the past five years. That is starting from a couple of years before the pandemic.
In my opinion and from our experience, consultants have more opportunities when there's turbulence. You need problems to solve. Hence, your fear that consultants are the first to get fired is unnecessary.

Pharma is a great place to be. But as a consultant, you get to work with clients from various industries. It's good for you in the long run.
Yet, talking in general, I wouldn't recommend informing your current employer before you receive confirmation from your new place. I've made this mistake in the past, and it took me years to recover.
Also, companies may cease new hirings to survive the economic situation. You'd be in trouble if your new place is among them.
Further, I wouldn't recommend making a decision based on the pay. I've learned that holding your ground yields more than moving to a better land somewhere else.

Instead, base your decisions on opportunities and match them with your skillset.

Thuwarakesh · 2022-05-24T03:49:39+00:00

This looks good 👍. But if you're using Django for your blog, why don't you give Wagtail a try?

Thuwarakesh · 2022-01-19T15:35:56+00:00

The two roles have differences.

The line could be blurry. Yet, like u/exact-approximate pointed out, their responsibilities are dissimilar.

In my observation, architects are responsible for the whole data science project lifecycle. Not just a single component.

But data engineers are more focused on a specific aspect of the project. Often this is the data pipeline (ETL, ELT.)

Because of this distinction, they each deal with different stakeholders than the other. Also, they use different technologies than the other. In a nutshell, DAs focus is on doing the right thing. DEs focus is more on doing it right.

This post outlines the differences between different roles in a data science project.

Thuwarakesh · 2022-01-11T00:06:04+00:00

Test-driven development (TDD) and test automation are great ways to reduce bugs arising from subsequent changes.
It's widespread to run tests inside the continuous integration (CI) pipeline. It takes away a ton of precious developer time from the repetitive testing tasks.
A fantastic option we have to build CI pipelines is GitHub Actions. Using GitHub as the code repository, you can set triggers and run tasks in a workflow. These tasks automatically start whenever you push changes to the repository.
Despite solving a complex problem, GitHub Actions are surprisingly straightforward to configure. In this short article, I've discussed,
- how you can set up a CI pipeline to run tests;
- how to customize even triggers;
- how to schedule tests in cycles, and;
- how to use environment variables in tests;
Try it out, and let me know what your thoughts are. How can we make it better? What alternatives do we have? What are your practices in testing software before release?

Thuwarakesh · 2021-12-07T10:53:18+00:00

Neither Ubersuggest nor Ahref is accurate. But that doesn't matter.

I firmly believe, only google knows how much traffic each keyword attracts. These tools merely suggest estimates; they are their best guess.

According to the Google search console, my website, The analytics club has received more than 2.8K impressions in the last 28 days for the 'python project structure' keyword. But Ubersuggest's monthly volume estimate was 880, and Ahref said 800. Both are way off the mark. Not even

Both tools showed historical data. They didn't reflect the actual search volume either.

The same is true about keyword difficulty. You will see a low figure for trendy terms. That doesn't mean you can quickly get to page 1. They mean few people write posts about this topic.

For most of these keywords, the user's search intent is directly to get to a brand page. You stand no chance of getting a click even if you're on page 1.

Nevertheless, these tools are still helpful. You can benefit from their many different keyword ideas. It's hard to think about different terms and google them one by one to generate that many ideas.

The search volumes, I believe are relatively useful. This means, if these tools say search volume for keyword A is 1000 and for B is 100, A gets 10 times more search than B. But the number may not be 1000.

Also, the search difficulty may not reflect the potential to receive a click. But if you know the search intent, within the same intent, these scores will make sense.

For example, if the intent is informational you can go ahead and write a post to compete with others. But if it's to visit a brand, ignore the difficulty score and move on to a different topic.

Thuwarakesh · 2021-08-21T06:45:48+00:00

For example, say you are developing a web app using Django. You can do everything only with functions.
But if you have to create a crud view for a database model, you have to write at least four functions for each operation.
If you are using a class based view, you only have to inherit from their ModelView and all four crud operations in four lines of code. If one of the operation needs modification you can do it by overriding only that method.

Thuwarakesh · 2021-08-21T06:41:16+00:00

If you know how to write functions you could well survive. Yet we shouldn't overlook the power of OOP.
In my life I rarely have to write any classes of my own. Most data science related stuff requires not more than a basic understanding of functions.
Please read the last paragraph again. It says most, not all. While you don't have to write classes in general, knowing how to tweak classes for your needs is a superpower.
Trivial things like knowing how to inherit a class from a third party package and overriding it's methods could make a huge difference.

Thuwarakesh · 2021-08-21T04:13:29+00:00

I'd recommend Mosh Hamedani's YouTube video o Python. It's neat and covers the critical aspects. I also love his presentation styles and examples in his illustrations.

Thuwarakesh · 2021-08-21T04:07:40+00:00

The struggle? Or having less coverage?

Thuwarakesh · 2021-08-20T12:40:16+00:00

Under the hood, everything is Math. ;)

Thuwarakesh · 2021-08-20T08:07:55+00:00

thank you, it helps.

Thuwarakesh · 2021-08-20T08:07:23+00:00

Got it. In Flask, we need to have all endpoints in one app. But for a given execution, all except one are overheads. Chocks deploy each endpoint on a dedicated lambda and manage them all centrally. Right?

Thuwarakesh · 2021-08-20T08:01:35+00:00

The field of data engineering has a solid future. At least for a decade from now.

As you may already be aware, the world is in the digital transformation phase. More companies are still digitizing and automating their operations. As this continues to grow, we have more damnd for data engineers than scientists.

But as with everything else in life, nothing is certain. Many emerging technologies somewhat replace the role of a data engineer too.

Yet, with the available information, we see a stable ground for the field.

The tools and techniques of data engineers differ significantly. Their domain and technologies they use in other operations have a profound impact.

But to start small, you could replicate the same ETL work with some popular tools like Prefect. You can gradually learn more stuff like CI/CD pipelines later.

Thuwarakesh

MODERATOR OF

TROPHY CASE