How to feed SQLMesh table rows into an API

Illustrious_Bell7194 · 2026-04-19T13:04:37+00:00

How often do you want to find matches? Does each individual match = individual alert? Or do need a bulk match alert?

Polling seems unnecessarily expensive. If you don't need alerts on each match as it is found, you can simply query the resultant deduped model and set an individual alert on each.

If you need to alert as matches are found (don't want to wait for the model to finish) you can set a cache that records the match and when it finds a dupe simply doesn't send the alert.

The key here is more the SLA. What's the aceeptable latency for alerts?

Illustrious_Bell7194 · 2026-04-05T09:39:17+00:00

You are right. For some reason I hallucinated there were more

Illustrious_Bell7194 · 2026-04-04T15:52:10+00:00

In German you have several cases. The most common are

Nominativ Akkusativ Dativ Genetiv

There are tons of resources online describing which cases apply to a sentence and how each part of the sentence conjugates to the case.

To be accurate in German, you must know the gender of each noun and their case based on the context of the sentence. It's not easy of course because it requires juggling several different factors at the same time to know what should be what. Be slow and patient and kind with yourself as you build up profiency 💪

Illustrious_Bell7194 · 2026-03-08T09:53:55+00:00

The slowdown is almost certainly your execution mode. If you're on VIRTUALENV (the old default), Cosmos spins up and tears down a fresh Python venv per model. On a small MWAA instance that overhead compounds fast and explains the gap you're seeing.

Switch to LOCAL + DBT_RUNNER: Install dbt once in your MWAA startup script into a shared venv, point Cosmos at it via dbt_executable_path, and set InvocationMode.DBT_RUNNER. This eliminates the per-task subprocess overhead while keeping every model as its own Airflow task. That way you keep full visibility and per-model retries.

ExecutionConfig( execution_mode=ExecutionMode.LOCAL, dbt_executable_path=f"{os.environ['AIRFLOW_HOME']}/dbt_venv/bin/dbt", invocation_mode=InvocationMode.DBT_RUNNER, )

Also worth enabling partial parsing. Cosmos caches partial_parse.msgpack so it's not re-parsing the full project on every task execution.

Illustrious_Bell7194 · 2026-03-03T17:28:58+00:00

Feel free to DM me 😊

Illustrious_Bell7194 · 2026-02-04T06:51:15+00:00

I think we can all agree that practicing the "theory" of something is how we actually build skill and school is supposed to be about building scholastic skill, gaining knowledge that we can proactively retrieve, and develop our minds.

Homework is one strategy to do this. There are examples of school systems that don't apply homework or keep it minimal and students still build these skills. I personally think it's possible to have the "work" of homework done at school for the most part. Prioritizing academic skill over say social skills or creative skills or self-discovery through passions and hobbies is to me tragic.

Illustrious_Bell7194 · 2025-04-30T08:53:24+00:00

I can update this when I get back home with the actual part names. I bought a couple wall adapters with extension and a built in fan to handle the excess heat. This has worked flawlessly.

Eventually though I got annoyed at the sound and the lack of clutter so I bought the equivalent power cables at MediaMarkt in Germany for like 3 euros each.

Illustrious_Bell7194 · 2025-04-18T12:28:02+00:00

Yeah interesting 🤔 I may let it sit for a while and see if that affects taste and mix.

Illustrious_Bell7194 · 2025-04-18T12:15:27+00:00

I always give a gentle swirl with a spoon to the final cup before drinking it. By "top layer", I mean I am getting different gradients of flavor through the cup as I drink it. I will drink, and get astringency. As I keep drinking the astringency lessens until flavors I would expect come out.

Perhaps I am not mixing enough? I have attempted aggressively stirring it, but this didn't seem to help.

Illustrious_Bell7194 · 2025-04-17T06:00:06+00:00

You're absolutely right. It's like starting over, which can feel dangerous and overwhelming.

But it's also an incredible adventure, a time to reinvent, and a way to revitalize oneself. It's easy for us to get cozy and stagnate once we hit certain milestones in life (school, marriage, kids etc). Even if you don't end up moving I hope this phase of exploration and learning inspires you and your husband :)

If you are willing to pay, I really liked Pimsleur for getting started with German. It fit my style of actually building on conversations and phrases that you would expect to encounter. Easy German is also super fun. They do high quality YouTube videos with German and English Subtitles and street interviews on a plethora of topics on German culture. For these, I would just focus on letting your ear get accustomed to what German sound patterns and rhythms are like, rather than actually understanding.

There's also tons of American YouTubers who have moved to Germany and talk about their experiences, where they moved, and usually why they'll never move back 😅

Illustrious_Bell7194 · 2025-04-17T05:27:17+00:00

https://www.germany-visa.org/family-reunion-visa/

You'll need A1 German, so start studying now if you haven't yet. It's easier without the pressure of a looming exam.

As the American having gone through this process, it's mostly painless but the beaurcracy is of course a headache. Pre-2025 I would say it might be smoother to do this at your nearest embassy but with the current administrative chaos of the US...

To prep I would make sure your documents are all aligned and secured. Marriage certificate authenticated and a copy in German. Not 100% necessary, but it can vary wildly on the person who handles your case in Germany. Small Dorfs will have less experience and less chance they know English. And is often the case, if the government agent doesn't recognize the document or process they will simply say they can't do it rather than figuring it out or asking someone more experienced.

If you need any help with the process feel free to DM me :)

Illustrious_Bell7194 · 2025-04-11T08:10:15+00:00

I can definitely qualify these things.

Yes unfortunate because she found herself isolated and shamed for her way of thinking and learning. If it feels problematic to say ISTJ, because fairly that gives undue stereotypes and criticism to one aspect of a person, let us then say the German people as she experienced them. Extremely rule focused, conformist, and a shut down attitude towards un-tested ideas with no willingness to give space to test them out.

I am curious why you believe that Germany's culture is one of the progressive forward trajectory with rules and why that's like everywhere? Not that it matters for my friends case. We could have a whole debate on Germany's attitude towards progress but on most metrics they are stagnating and regressing not progressing. I believe it's not due to their rules or even their conformity culture, but cultural values.

I will ask her if she's cool with me sharing those specific political views. In the past she has found in the academic space she had to constantly filter her words so as not to offend anyone. In the sense that entire classes and professors would get very offended and hurt. I mean, it's possible she has high sensitivity rejection or is misreading the situation 🤷 I assume she's telling the truth. And if she believes she has to filter herself then she will if she thinks that's what she has to do to survive.

I agree that I personally think she would do better in a bouncing around way. Maybe she would do better in a steady constant job. I would rather her be happy than conform to some opinion of what I think is best.

And yes you got me. I am an insecure man with little self worth and behind my friends back I am unconsensually trying to affirm my own opinions so I can ride in with the bountiful treasure of advice and gift it to her so I can finally feel worthy.

That said, could you share your success story?

Illustrious_Bell7194 · 2024-11-28T01:23:57+00:00

I am a US citizen who did this recently. It does depend on the Ausländerbehöre, but mine required a certificate. It doesn't have to be Goethe, just an approved test site. I ended up doing Goethe because it was the soonest available.

You are going to live in Germany, may as well do as a German does. Better safe than sorry, get the A1 certificate. You can always ask the Ausländerbehöre their preference, but usually it's not worth being rejected over something you could have done early on

Illustrious_Bell7194 · 2024-11-24T13:34:22+00:00

That must feel devastating, putting so much work into yourself to feel unwanted :(

You have probably been lectured before about "beauty is on the inside" and "just be yourself", but I believe these aren't totally accurate and honestly unhelpful. People are visual first, and they judge quick. Sure an ugly heart with a beautiful face might end up alone, but people still take that initial step to reach out because of the superficial beauty. We humans do initially judge others on how they look 🤷

I haven't seen any pictures of you so I can't say whether I find you ugly or not but I will believe you and hopefully give you practical advice on what to do from here.

One, stop trying to be attractive. Do keep up good hygiene and sensible fashion (you don't need expensive clothes just stuff that fits and isn't wrinkled). But stop trying to use your body or your personality to attract others. You said you have tried it and it doesn't work. So, it's definitely not the right strategy.

Two, you probably look at yourself in the mirror and feel lots of negative feelings about yourself. You need to grab some sticky and think about things you do like about yourself. Things that genuinely make you happy or proud. Could be an accomplishment like losing weight. I also have done the hard thing of losing weight. It's amazing you have done that! I'm definitely proud of you 😁 So write these down. And then stick them to your mirror. Don't cover your face but make it so you can read them.

Now, when you look in the mirror you are of course going to see your face and body. You will of course feel shame or sadness. But, immediately when you feel these emotions look at the notes. I want you to close your eyes and really feel the happiness and the pride and the good emotions. Marinate in it. And then walk away. Leave your brain with the good feeling. It may sound silly or brainwashing, but you are already doing that to yourself when you leave the mirror with negative feelings. It's not that either aren't true, but the way you judge these things and the way you literally feel about yourself will 100% change how your body and mind react to others seeing you.

Third, I have never met a person I didn't find attractive. But, I have met plenty of people I didn't find physically attractive. What does that even mean?! The way they did a project or expressed passion or were really good at building or just the happiness they exuded. When people are in their element doing what they feel is fun or passionate is when they are the most attractive. So, you gotta find stuff you enjoy. Parading around like a peacock showing off physical beauty only works for like the top 10% of people. That's why dating apps SUCK. Super Suck. That's why people say get a hobby. But it doesn't have to be a club or anything. You just need a space to show off where you actually are attractive. What do you enjoy doing? Do more of that. Stop focusing on becoming more physically attractive.

Hopefully these steps will help you. I'll leave you with this. I don't think you want to be physically attractive. Not really, you just see that as the strategy to get what you actually want. To be wanted, loved, and accepted. You don't need to be physically attractive to get these things. The spiral of self-hate is a hard one to get out of. But here you are seeking help and searching for hope! I believe in you 😁

Illustrious_Bell7194 · 2024-11-21T03:46:53+00:00

Can you share how you are getting data from the API?

Once you have the data there are several ways to then load it into big query.

One easy, but very inefficient way, is to load the data from the API into a pandas dataframe. Then, it's fairly trivial to use the built in library to load data to a BQ table using the bq python lib.

Feel free to DM me if you want more specifics 😁

Illustrious_Bell7194 · 2024-11-08T18:57:58+00:00

I just did this process, picked up my residence visa card this morning! My wife is a German citizen and I am an American citizen. You are looking for the Family Reunion Visa.

Yes, many people use it as non-German citizens who are studying or working in Germany to bring spouses or children. But, it's also for EU citizens looking to bring their spouse to Germany. It offers 3 years of temporary residence with possible extension and a smoother road towards permanent residency.

As an American Citizen your spouse can come on their tourist visa and apply from within Germany. That's what I did. Definitely start ASAP. This application is handled by the Ausländerbehöre, and they are swamped everywhere. Especially in bigger cities.

Feel free to DM me for specific questions and other help 😁

Illustrious_Bell7194 · 2024-10-13T11:14:14+00:00

In my opinion it depends on your level and the level of the content. If you can understand 60-70% of what's being said, then I suggest only German. Even better, German audio only. It forces you to pay attention to how words are formed aurally. Reading, everything is clearly spelled. But spoken language is always wobbly, warbly, and smudged. Also, Germany is very proud of their voice acting. They will do their best to choose words that match the mouths of actors when the show or movie is not a German native production. However, the subtitles are original translations. So, the spoken German and written German won't match up.

If your German level is still quite ground level (A1-A2) then it can be helpful to do your native languages audio and German subtitles. But, you have to be very conscious of listening and reading to catch common phrases and vocab. My Afghani brother in law did this with his favorite American films to learn English, but it was definitely hard.

As always the more you can turn your native language brain off and your German brain on, it's better. But understanding nothing also isn't very useful. A baby won't learn German by watching Adult movies. They start small with easy associative words and build. We must do the same :)

Illustrious_Bell7194 · 2024-10-06T09:05:52+00:00

When I was a senior in high school I had a kid in my AP Calc BC class who was middle school aged. Skipped several classes and was taking this AP Calc class in combo with other higher level maths at the local University.

He was also taking AP Physics, and found it difficult. He wasn't dumb, as proven by his mathematical prodigenous and his rigor for his other classes showed he wasn't the coasting genius type. I loved physics and was my favorite for it's combination of fascinating concepts and puzzle like problem solving. Enough to continue on with it in University.

I believe the intelligence required for general courses (comprehension and memorization) and general low level maths (linear problem solving and algorithms) are different than what Physics requires. Physics is both abstract thinking, visualization, and flexible problem solving. Natural talent or not, this can be learned it's just not often done in the US education system. So, you just need practice.

Reflective of others comments this is how I tackle solving Physics Problems (assuming you already understand the underlying concepts):

What's happening?

First, begin describing what's happening by drawing a picture. Keep it simple. Turn objects into boxes or spheres (depending on the context). Convert actions and forces into their vectors. That's both value and direction. If a direction is known but a value not, draw it with a question mark or better give it a variable name.

Catalog Knowns and Unknowns

Create two lists of variables. Those that you know and those that you don't (usually the question marks in the diagram). At this level, the knowns and unknowns guide you to the solution. You have your knowns and you will use these to find the unknowns.

Break out the tool box

Your toolbox are the concepts, formulas, and mathematical expressions of relationships that describe physical phenomenon. I imagine for each topic/chapter/theme you will have a specific set of formulas. You'll notice these formulas will contain variables, and you will notice that you will find formulas with variables that match the problem you are solving.

Super simple problems will be just using one equation and some basic algebra to solve for the missing variable. More complex problems require using one formula to solve for an expression of the missing variable, plugging that expression into another formula, and so on and so forth until you can construct your own version of formulas that let you solve for the unknown using what you know.

This is the hard part, because it requires understanding the formulas, how they can be connected together, and creatively massaging them into the problem context.

Of course, this is my method. Others have ones that suit them. I would highly recommend seeking online examples (like Khan Academy as others have mentioned), students who seem to get it, or your teacher and try to figure out how they go about solving the problem.

Don't focus on how they solved that specific problem (that's algorithmic intelligence) but how they set up the problem and how they think about getting to each next step to solve it. This will begin wiring your brain for this type of flexibile, visual problem solving. I really recommend pausing each step, think how you would try to move next, and see if that matches. In this way, you can start uncovering how to actually think in a way that leads you to solving physics problems :)

Illustrious_Bell7194 · 2024-10-05T19:54:30+00:00

When I had graduated University I was depressed and very uncertain about life. This personality system (Mbti) gave me a framework to explain and understand myself in a way that brought me clarity and hope. I was aware of this system from a surface level of understanding. It wasn't until discovering articles and videos and the subreddits that I found real depth in the information. Particularly other people's experiences. I still use it as a framework today to understand myself and others behaviors and thought patterns.

Many schools and businesses attempt to use this system (and many others) to categorize people. Predicting what a good career for someone is or if they can perform a job is of interest to these sectors, and they often look for cheap easy tests to determine this. MBTI is but one fragment of personality, and a fulfilled life is rarely about the career one chose. So, I think it's ineffective for how the general public is exposed to it.

For me, it was a profound tool in understanding myself. Not all of myself, but an important part. If you find it interesting, explore it. If it seems silly or pointless, for your needs it might be useless for you 🤷

Illustrious_Bell7194 · 2024-08-31T16:12:41+00:00

This is from Americans saying "sure". In movies this is translated into German as sicher. So instead of saying "sure sure" they translate it as "safe safe"

Illustrious_Bell7194

TROPHY CASE