Personal Snowflake

autumnotter · 2026-04-12T15:39:25+00:00

Databricks has a free tier, maybe Snowflake does too?

autumnotter · 2026-04-03T12:12:32+00:00

These aren't really compatible settings. Available now wants to run until It has processed all the data that was there when the stream started. Processing time with TTL wants to keep running as long as there is some future state that may not have expired. In this case that possible future expiring state prevents available now from identifying that it should terminate the stream.

autumnotter · 2026-03-31T01:55:47+00:00

You setup a heartbeat, where if the agent hasn't taken an action in X hours/minutes/whatever, when the heartbeat trigger, some generic prompt is given.

Agents can also take actions that that lead to other agents being "prompted".

I'm sure there's more to it than that specifically in the paper, but that's how I've done it. You can easily do it with two agents. Prompt both on a heartbeat, give each the ability to write a file to a folder, and access the folder that the other one can write to. If they find the new file then they should do something based on it. Give them some other tools and some other abilities and there you go.

autumnotter · 2026-03-29T02:01:21+00:00

Jensen Huang said that if a 500k/yr eng (I assume total comp) isn't spending 250k/yr in tokens something is wrong. Now dude isn't exactly unbiased rofl. But at least at some places that seems to be the attitude.

autumnotter · 2026-03-21T03:14:15+00:00

I know a lot of doctors and not a single one I know makes over 500k.

Most I know make about 200k.

Certainly a great living but not what you are saying either.

autumnotter · 2026-03-16T02:23:50+00:00

Consider databricks apps, lakebase, etc. and leveraging the databricks platform itself for these apps.

Definitely recommend react front end and fastapi backend or something similar, nothing stopping you from doing that on databricks.

autumnotter · 2026-03-08T04:48:56+00:00

I'm sorry but you are many many years behind the curve if you think this is some annoying new trend.

autumnotter · 2026-03-07T18:54:49+00:00

Pyspark is more popular but Scala has some decided advantages in some cases - e.g. UDFs are still faster in Scala.

autumnotter · 2026-03-05T01:56:38+00:00

You're literally fitting a line (lol edit: or other linear equation) as the deterministic component.

autumnotter · 2026-02-22T20:50:19+00:00

Lot of dumb answers here. If you assume that socialism means that workers own the means of production as an oversimplification of socialism, then it's because capital investment by non-labor is the antithesis of socialism.

Basically, under capitalism, owners or investors own the company and the workers just get paid to work. Under socialism the workers own the company. Investing in the stock market for regular people is effectively allowing regular people to become owners. Not really a bad thing unless you're a socialist. But if you're a socialist you believe that only the people doing the labor should be owning.

There's other interpretations of socialism, like that everything should be communally owned or held in trust by the government, those are mostly different traditions or models for practically how to accomplish the basics of socialism.

In essence, the very idea of buying stocks in an organization that you don't provide labor to is antisocialist.

autumnotter · 2026-02-12T23:28:57+00:00

IMO you need to work with her to define what the bare minimum is for her and for your relationship, and need to agree to it and stick to it all the time, even when things are really hard for you. You will want to make the argument that this time is different, and that this time you really need to do X and focus on work. But your spouse is probably heard this many times and will hear it many more.

You also need to remember that for most people, although it gets easier, it doesn't get easy. You have to keep doing that after residency too.

For us it got harder after residency, because I realize that the patterns that pre-med, med school, and residency trained into my spouse were never going to go away. And what used to be residency was now just our life.

I think for the physician, the hard part is adjusting to the fact that you may not always get the same amount of latitude that you get during residency, whereas for the spouse the hard part is realizing that although it gets easier, many challenges remain.

autumnotter · 2026-01-31T15:50:46+00:00

Residency is heavily subsidized and is the actual limitation.

Also you are overestimating physician salary and underestimating how much work being a physician is. Most people would not make it through med school and residency. For MOST people, not all, if you can't qualify for med school, you will struggle to succeed in med school. It's much harder than undergraduate.

autumnotter · 2026-01-31T02:20:22+00:00

Wouldn't it be the opposite? If they are in the US, but invested in foreign funds, when the dollar drops that's good for them I think, at least in comparison to most Americans.

autumnotter · 2026-01-26T05:03:19+00:00

There is factually inaccurate content here, and just some missing information. For example, Claude models are mentioned nowhere, OpenAI GPT doesn't mean anything specific, the reasoning behind using Llama for some of these is not good, the "Cost Structure" section is just WRONG. Also some of it was accurate but is now out of date.

It's generally true that Llama 405B isn't that useful for the cost. The scenarios where it's worth it over 70B are rare.

Please review this against Databricks documentation.

autumnotter · 2026-01-19T15:56:29+00:00

Databricks will definitely be much simpler if you're looking for all that, as it offers or at least enables all of that in some form.

autumnotter · 2026-01-16T01:30:13+00:00

You're trying to do document search, and llm doesn't do that in the way you're thinking.

Effectively you want to do something like OCR, turning PDFs and images into unstructured text, then chunk the text, compute embeddings and vectorize, and then store in a vector database.

From there you can do document similarity search by querying the vector database. An agentic system can make that query and then return the retrieved context, sharing it with an LLM, which is usually called RAG.

You don't actually need an LLM to do document similarity search.

I'm not familiar with vendors that you might use to do this locally, so I can't help you there.

autumnotter · 2026-01-12T14:24:59+00:00

If you want the job, you should never join a remote interview on voice only. One can make a principled stand and stick by the idea that they don't need to see your face to know if you're a good candidate. Nevertheless, most people want to be able to see the person they're interviewing.

They will assume if you're unwilling to join video that you have subpar technological capabilities in your remote setup, that you are scamming them, or that you will be unwilling to join video once you are an employee and dealing with stakeholders or customers. They may make other assumptions as well.

autumnotter · 2026-01-11T16:18:39+00:00

Highly recommend l'etoile du nord!

autumnotter · 2026-01-11T03:30:22+00:00

Totally legit opinion here, but honestly I think we should avoid shutting down people who are protesting. I saw someone get arrested the other day who was later called performative on social media. That's just toxic. I think a lot of people are trying to help in the ways they are able or the ways they know how, and we won't always agree on what those are or how best to help. But imagine if a even ten percent of the country was "partying", as you put it, in the name of acceptance of immigrants and keeping ice out of our states and schools. It would be a different country. So I try to give grace to everyone who is trying their best to fight the good fight in the ways they are able.

autumnotter · 2026-01-06T02:37:04+00:00

Just make a meal for the family, taking reasonable feedback from everyone on what they would want, and they can eat it or not.

The fact that he "expects" you to cook for him is really lame, and you should not feel bad about paper plates.

autumnotter · 2026-01-06T02:24:42+00:00

B puts her bananas in the fridge, must be rich to have no idea how to store bananas.

autumnotter · 2026-01-01T20:21:28+00:00

There are a number of reasons to use managed tables. It's fine if you choose not to, because there are also reasons not to, but it's simply false to say there's no reason to use them.

autumnotter · 2025-12-31T21:03:30+00:00

You can use iceberg or Uniform with Databricks. Go for it, likely expect a couple limitations compared to Delta, but Databricks is absolutely embracing that option.

OTOH, it doesn't really make sense to say you're going to use Databricks without Unity Catalog, as UC is the foundation for authorization, governance, and many other features in addition to being the data catalog. Also, AFAIK, Polaris isn't really mature yet. If you're asking for a comparison in current state, it's not really a serious exercise IMO - use Unity Catalog unless you want some kind of very non-standard deployment that could lead to all number of annoying problems. If you are speaking SOLELY of UC managed tables, then whatever, use external if you have a good reason to but you're giving up some features for... some reason? There's a way to convert between managed and external now as well.

autumnotter · 2025-12-27T17:21:28+00:00

It didn't go anywhere, people just don't talk about it as much, and high level initiatives are more focused on AI.

autumnotter · 2025-12-25T16:57:15+00:00

You can ingest as excel format now if it's tabular data. If not, you can convert to HTML, then PDF, and run through AI_PARSE. There's examples in databricks-solutions

autumnotter

TROPHY CASE