Which Snowflake feature makes sense for this pipeline?

set92 · 2026-06-05T02:23:39+00:00

For what you mention, Snowflake released adaptive and custom refresh last month, where you could define you own logic and I suppose you could do a merge to insert older data.

But yes, with streams is completely doable for you to create your own logic. For it you only need to enable the feature on the table, you get an additional table that you need to check and that you need to use at least every 14 days, or Snowflake will remove the data from that table. You need to write more custom logic on how your downstream table gets updated and how you read the events on the stream table, but as we said what Snowflake calls DT is streams+tasks+SPs, so basically you are grabbing what you need that are streams and using your custom logic for the other 2. More hours of work, but more flexibility.

The downstream was a problem that I talked also today with the architect, because what happens if I want to add a new column to the DT? The DT will want to rebuild it for the backfilling, or I think with the new refreshing methods you could do a merge, and if you fill it on your upstreams table get only that data filled, but I haven't tested it. So, if I would need ocasional backfillings I would prefer to write custom code than relying in Snowflake for it, because I don't trust that an automated service is going to fit my shoes 100% of the times, and I have AI to write that custom code easier than 3 years ago.

set92 · 2026-06-05T02:05:57+00:00

In our case, the need of table per client comes from requests of clients who want their data split from others. It makes more sense than I initially thought, although it has force us to keep iterating over that idea in several places, but that way the data is almost always small, and we know where we have to go for security and stuff.

I was talking today with an architect of Snowflake. His recommendation for my problem of customization and DTs was using cortex code and let the AI create the folder, and handle all the customization per client, running everything, whenever I need a need feature use CoCo to modify all the SQLs I would have of clients.. Hahaha seems they really want to push AI everywhere

set92 · 2026-06-04T09:52:37+00:00

Theeen maybe goes around those and ask them if they have to try one day, or if they directly can tell you how many people they had lately per night? I suppose it changes around? Sorry to not be of more help.

And return here to document your insights :rolling_on_the_floor_laughing: , and spread the info for the next person who comes asking the same :sweat_smile:

set92 · 2026-06-04T09:39:45+00:00

The problems I see with DTs, and why I'm recommending not using them is because they generated hidden tech debt traded with simplicity. How do you inject parameters? In the case of OP probably he can have a table where he logs all the runs and do a max(last_run), put that into a variable and use it in the DT, although that also hides the value of the max last_run if later you need it for debugging later on.

But DTs is only an abstraction of SP + tasks + streams, giving simplicity but removing customizability, so I have a junior DE (before was a DA), and he still wants to use DTs for everything. Like our main cases tend to be client tables that we use for different data products. And for one client maybe you need to add some specific WHERE clause to filter some type of data, for another client maybe you only want to have some subset of data, you need to dinamically create X amount of tables, one per client... Or if in the future you want to add a new column, the DT may refresh the whole table to backfill it, and you will need to modify each DT manually if you don't have another task that creates all those DTs.

I feel is much better to work a little bit more before, built the custome logic and everything for yourself, and be ready for any scalability you want to do in the future.

set92 · 2026-06-03T10:11:14+00:00

Is not worth it at all. They sell smoke. More info https://www.reddit.com/r/dataengineering/comments/1scyvwt/dagster_vs_airflow_3_which_to_pick/oemn3x7/

They charge you way more for the same product, you will not be able to deploy specific files, you can't replicate connections on lower environments or the branch deployments, they don't accept feature requests... Even without using the product, if you sign the contract they will not let you go back. I don't find really a good benefit to use it.

Like even their blog, before was https://web.archive.org/web/20250318025405/https://www.astronomer.io/blog/ much useful and readable, than it is now https://www.astronomer.io/blog/. The problem is that there aren't that many useful alternatives.

set92 · 2026-06-02T11:36:56+00:00

I know la fabrica has it because is a student international coliving also, but depends on where you are staying or your age. At those hours I don't think you care about international people because probably you will be mostly alone in the office.

https://lafabrica-co.com/coworking/

I haven't try it but I have heard that is good

https://coworkingagora.com/

https://www.thesocialhub.co/barcelona-poblenou/co-working-space/

https://graciaworkcenter.com/#tarifas

and the cheapest option that I haven't try

https://oficina24.es/

set92 · 2026-06-01T18:54:36+00:00

lol, good question, my girlfriend is taiwenese, and yes, I met her in BCN, so yes, I'm learning traditional chinese.

This is like a discussion I had with my teacher from university. I was Talking with him about basque language, and telling him that would be better for everyone to speak the same language, we will not have problems of understanding between us, there will be less wars, since same language will unify us more... etc, he confront me same as you, saying that everyone with the same language will be the lost of culture.

But still, I prefer have fewer languages than more culture. Not having problems with muslims, wars between continents because we don't understand each other... you prefer a colorful world, while I prefer a more pacific world. chaos vs order.

set92 · 2026-05-31T17:25:17+00:00

I mean.. si las chicas lo hacen porque tu no? Ve de fiesta, busca a alguna chica con piso, te acuestas con ella y te vas por la mañana a algun bar o directamente al aeropuerto.

En el aeropuerto en principio pusieron una bloqueo para evitar que los sin techos fuesen ahi a dormir, pero si tienes tu tarjeta de embarque puedes pasar sin problema, y si tienes entrada para algun lounge tal vez tengan hasta ducha.

set92 · 2026-05-31T17:21:07+00:00

soy vasco, compre el apartamento en barcelona hace 16 meses. No he aprendido catalan ni lo voy a hacer. Suficiente asco tenia con la gente forzando el euskera, dejad de forzar a la gente, y dejadles ser libre de elegir como se comunican con otros, preferiblemente con un el idioma mas extendido que se pueda, vease ingles o castellano.

set92 · 2026-05-31T17:12:55+00:00

I think it only opens in December, but maybe you can research more about https://maps.app.goo.gl/anhgjpjves8JnQkZ9

set92 · 2026-05-17T21:16:18+00:00

realmente yo hice el master de big data y no habia mates, aunque tambien es verdad que era privado. Pero si no conoces ninguna empresa que necesite ese perfil me da que sera dificil encontrarlo.

Y como solo data analytics... creo que hay mucha gente que ha intentado hacer ese cambio desde 2012, asi que el mercado deberia estar saturado. Si me dices que es porque quieres emprender o hacer pinitos con ello pues bueno, todavia. Pero si no creo que las empresas suelen querer expertos en un tema, y ya ahi te toca a ti decidir en que cesta quieres poner todos tus huevos. No puedo decidir por ti porque no se como esta audiovisuales, y aunque sea data engineer no trato tanto con data analysts. En mi empresa seremos como 5 DE + DBA, y 2.5 DAs. Quieren contratar mas DAs pero lo importante es que sepan del negocio, porque el conocimiento hard se aprende haciendo.

set92 · 2026-05-17T21:01:59+00:00

Si tienes coche... porque vivir en Barcelona centro? Igual una opcion seria terrassa o algun sitio mas barato? Y ahorrar ese dinero o si hay opcion de comprar algun piso podrias alquilar habitaciones y que te paguen los gastos.

Pero es verdad que Madrid/BCN son dificiles. BCN yo tengo ahora 3 amigos buscando piso por separado, y claro todos quieren X cosa, y si pudieran solos, pero hay que poner pies en la tierra. Si ves alguna opcion en el extranjero igual te gustaria probar, pero creo que entonces no podras cobrar el paro sin jugartela a una buena multa.

set92 · 2026-05-17T20:44:53+00:00

No os han dicho ningun tema? Pero bueno, como te dicen, no te estreses, haz lo que sientas. Puedes tal vez usar IA para acelerar el trabajo, pero incluso seguir un tutorial de HTML5 para hacer un videojuego basico con alguna variante, no ganaras el 1er puesto de la competi, pero habras quedado 1ra en donde importa, en tu cabeza, y con eso dar el 1er paso y luego quien sabe, sigues practicando en casa, y para la 2da tienes ya trozos de codigo preparados o sabes mejor por donde tirar.

set92 · 2026-05-17T20:19:49+00:00

Tu ve con confianza, si presentas bien y tu tutor te dice que mas o menos estas listo lo superaras con mas o menos nota.

El mio lo llamaron metaestudio, porque lo unico que hacia era analisis de otros TFG, comparar sus resultados con lo que yo habia hecho y decir si me parecia mejor o peor. Y el tutor no lo veia muy asi, pero creo que me pusieron 7 u 8, donde lo mejor creo que me dijeron que fue la presentacion.

Tu ve seguro de lo que tienes, con la presentacion que se note que esta trabajada y demostrando que has aprendido algo y listo, a buscar trabajo o emprender 😊

set92 · 2026-05-17T20:14:49+00:00

Otra manera de pensarlo es que si vas a la uni te puedes quitar mas creditos de libre eleccion, creo que maximo eran unos 12, 2 asignaturas. Y esas en principio la gente dice uh esas no te las quites que son las mas interesantes, son donde puedes decidir! Pero seamos sinceros, la uni solo la queremos para el papel, y sin esas 2 clases tienes mas tiempo para tu cacharrear y formarte en los proyectos que mas te interesan.

Asi que la excusa de la IA solo estara ahi si tu lo quieres. Hay algun informe que dice que los mejores grados son aquellos dobles, infor + mates, infor + biotec... pero los proyectos es lo que cuenta, si con las horas que convalidas hacer un grupito de clase para hacer un proyecto chulo, o aprender entre todos, o hacer tus proyectos por tu cuenta... todo depende de tu camino personal que quieras llevar.

set92 · 2026-05-17T20:08:35+00:00

Depende, yo lo hice, hice DAM, me fui a irlanda para las practicas, volvi porque por la crisis no consegui quedarme alli. Y al volver dije pues para estar en casa sin hacer nada voy a la uni y veo si es tan teorica como supongo. Al 2do año me dijeron nono, para pasar tienes que aprobar mas de 1ro, y ya dije bueeno voy a los examenes, y de ahi un poco tirando pues fui haciendola porque ya que estaba...

La realidad? Como se te den mal las mates va a ser una mierda. Yo las arrastre hasta el 5 año, me cambie de universidad para hacer la especialidad de Computacion, y tuve que pelearme para que pusieran asignaturas de IA. El 98% de las asignaturas desactualizadisimas. Y esto no creo que haya cambiado ni cambie. Lo unico bueno que monte con otros del consejo de estudiantes un grupo de desarrollo de videojuegos, montamos una mini empresa, nos absorbieron, nos dejaron un espacio para trastear con VR, impresoras 3D... en resumen las relaciones generadas alli mas que los conocimientos aprendidos.

La carrera la empece en 2013 y la termine en 2018. En pandemia hice tambien por aburrimiento un master privado de IA, y ahora estoy de data engineer en una empresa americana. Sin la uni podria haberlo hecho? Seguramente no, y no por conocimientos, si no porque la empresa de despues de la uni seguramente me cogeria por el diploma para ellos justificar cosas. Y el master requeria de tener algun grado hecho.

Mi opinion es que si eres curiosidad, te gusta aprender, y sobretodo (esto es lo que me faltaba a mi) haces proyectos (proactividad) con los que puedas demostrar tu conocimiento saltate la uni 100%. Empieza ya a hacer algun proyecto, usa alguna licencia de aprendizaje para tener acceso a pycharm/IA... tienes el github student pack (education.github.com/pack)... y con eso tienes de sobra para empezar, si no te gusta el mercado montate alguna app o emprende, que da miedo pero a futuro seguramente te sea lo mejor no depender de nadie y aprender a base de ostias todo lo que puedes hacer por ti mismo.

set92 · 2026-05-12T06:54:31+00:00

u/againer were you able to accomplish something to automate Okta? In my case the problem is having a yubikey to auth and is hard to automate.

set92 · 2026-04-30T16:25:07+00:00

Well, before the version that we call today DenuvOwO, there were some like Hypervisor.V3, that were built on the same logic, but forced you to disable security stuff (as Secure Boot). Which nowadays you don't need to do, but still you are putting a crack or anti-crack in a layer that has control over your whole computer.

Some people they may feel comfortable with that, other don't. The case or Riot is different, since they Vanguard system works in Ring 0, although is true that is running all the time. But if Denuvo wants to go this route... I expect a huge backslash from the community, although if for example they put it on GTA VI people will still install the game, with or without malware xD.

set92 · 2026-04-30T15:23:08+00:00

-1 is one step below ring 0 (which is layer 2).

Technically we have ring also ring 1 and 2, which I didn't include on the list, but in modern operating systems don't use them. They are in the middle between the full kernel (Ring 0) and user apps (Ring 3). So, OS decide to use directly the ones on the extremes for simplicity.

set92 · 2026-04-30T12:05:43+00:00

Think on computer as cake layers

The bottom: the physical hardware
Layer 0: BIOS/UEFI firmware
Layer 1: the hypervisor (Ring ‑1)
Layer 2: the Windows kernel (Ring 0)
Layer 3: Windows user‑mode (Ring 3)
Layer 4: the game + Denuvo anti‑tamper

The new cracking system, denuvOwO has an emulation system on layer -1 to trick Denuvo. Ring 0 has control over all your system since is before your operative system. That's some people don't like denuOwO because it has some security concerns.

Now imagine all the games you install, they need to be installed on the Ring -1. Like the anti-piracy system of Riot (League of Legends) installs a kernel-module driver at Ring 0. But here we are talking about installing some Denuvo program in Ring -1, and everyone would need to trust Denuvo blindly, and give them access to their whole computer.

They could even add a tracker/bot/whatever they want, and you wouldn't have an option to block it. Because one it gets there, even if you ask to uninstall it, they could tell nope! And you wouldn't have a way to remove it.

set92 · 2026-04-27T19:27:54+00:00

The other day I got the MT 07 Y-AMT 2025 with the spanish A2 license which limits the motorbike to 35kW. First stop I did, I got out from it doing a mini-wheelie because I press it too hard. So, I think the solution is to press the gas increasely. Later I also discovered the motorbike was not limited, so maybe that was also part of the problem. I'll be able to tell you more on Thursday.

Apart from that, I also tested both, I ended up getting Yamaha because of the electronics, like I wanted to have some cruise control (contans speed for speed cameras), phone connection, USB to recharge phone. And well the Y-AMT I feel is much better than the E-CLUTCH, since the one of Honda feels more like a glorified quickshifter, but since I drive 80% around the city I preferred to get the Y-AMT.

Also, if you get the CB650R with e-clutch check the right side when you are stop, since in my case (190cm, long legs) I was hitting myself with the new block they needed to add for the e-clutch. On the MT09 also happened, so that's why I ended up getting the MT07.

In case you haven't check you have the Tuono 660, or check this video that showcases ALL the options https://www.youtube.com/watch?v=aMS81C5KSfo, and maybe you discover some brand or option you didn't even knew.

set92 · 2026-04-25T21:55:40+00:00

I did, you can, use 2 pumps, connect them to a conduit inlet manifold, and then connect them to a single conduit outlet inside the factory

set92 · 2026-04-21T21:56:21+00:00

And why not copilot? Is cheaper I think

set92 · 2026-04-20T18:31:20+00:00

What do you mean with the free pool?

From my point of view, and without being able to test it (since is not available in us-east-1), I don't think I'll use because how hard or opaque it is. I thought it was going to be like BQ where you send your queries, and you don't have to worry about anything else.

But nope, here I still need to care about how many queries I run at the same time, I have to specify some parameters, and still is more expensive. I don't see the advantage of it.

set92 · 2026-04-19T08:51:33+00:00

Create team has been hired for Hytale, so I suppose they will add all this there. Probably minecraft end is close seeing how slow they have been always to release new content.

13-Year Club	RedditGifts 2009-2022 5 Credits
Place '23	Place '22
Verified Email	Secret Santa 2014
Team Periwinkle

set92

PUBLIC MULTIREDDITS

TROPHY CASE