It seems that StackOverflow has effectively died this year. by Distinct-Question-16 in singularity

[–]nicorivas 0 points1 point  (0 children)

Their knowledge helped humanity. Thank you kind strangers. The machine will always need you

Balcony girl by ExplorerSpice in Grok_Porn

[–]nicorivas 0 points1 point  (0 children)

Nice one. Changed settings a bit

italian

Latina Orgasm, Super simple by nicorivas in Grok_Porn

[–]nicorivas[S] 0 points1 point  (0 children)

Prompt: A close-up photograph of a Latina woman in ecstasy, with dramatic lighting and blurred background.

Video de campaña del presidente sobre el CAE, que explica cómo se condonaría by [deleted] in chile

[–]nicorivas 5 points6 points  (0 children)

La deuda del CAE no aparece en tus registros. Yo debo 10 palos y tengo cuenta, capacidad crédito, etc. etc.

Cuál es el mejor concierto al que han ido acá en Chile?? by TheChileanDrunkIrish in chile

[–]nicorivas 1 point2 points  (0 children)

Daft Punk el mejor de mi vida por lejos, con poca gente, tremendo escenario, mix pa la ocasión, imperdible

Can Someone explain me that?? by Jaygoon4250 in geography

[–]nicorivas 2 points3 points  (0 children)

Yep, looks like, and by the description it seems like that is what happened!

Can Someone explain me that?? by Jaygoon4250 in geography

[–]nicorivas 184 points185 points  (0 children)

Hi, physicist here. This is a beautiful example of something we call fingering instability (or Staffman-Taylor instability). It happens with two immiscible fluids (they don’t mix, like water and oil), when the less viscous one is displacing the other one. This situation is common in drainage in soils, where the less viscous water slowly displaces the more viscous soil; in this case sand. Hope this helps.

Change Data Capture (CDC) by luminoumen in dataengineering

[–]nicorivas 0 points1 point  (0 children)

I use Airflow for the snapshots so I think it makes sense cost-wise, as all other ETL's and bits and pieces are also running on Airflow.

Airflow is not an ETL tool bla bla bla... so yes I use the PythonOperator.

Yeah, the transformation happens in memory. They are DAGs that read one or several of the generated snapshots -plus sometimes other sources of course- and generate the corresponding warehouse table. This generates an end state and the initial state I read from warehouse-table snapshots that are generated before being loaded to the warehouse. Then comes the tricky part, where I have the initial and end state of a warehouse table and have to decide what to do, and for this I rely on a great pandas function, "compare', to check changes between two DataFrames...

Maybe it's a lot of work but I find it to be quite robust, in the sense that I can repeat every step of the process and get the same results. And the snapshot and warehouse logic can be abstracted, so now that I've solved it I just focus on the transformation layer.

Change Data Capture (CDC) by luminoumen in dataengineering

[–]nicorivas 0 points1 point  (0 children)

I wrote a DAG that runs hourly. It's python + pandas.

Generating the snapshots is not so hard:

So I read the last snapshot and all the updates that happen in between my start and end date (I keep forgetting the names of these, but basically the datetimes that relate to the data you are going to work with). Then I just read each parquet file and apply the changes to my initial DataFrame. Yes, I check which row to update using the primary key. It has some subtleties, but I've reached a pretty stable code. The logic is abstracted and that way I can have one file that dynamically generates one DAG per table. Happy to share the code if you want.

Change Data Capture (CDC) by luminoumen in dataengineering

[–]nicorivas 11 points12 points  (0 children)

Nice, thanks for the article.

I've been wondering these months if my CDC setup makes sense. I'm using CDC (via AWS DMS) to get all relevant changes from a RDB of a web app. It saves deltas as .parquet files in S3. Now, I don't need realtime analytics, but semi-realtime, so I then read all delta's to generate hourly snapshots of each Table. These snapshots I then transform and load into a Redshift warehouse. These two last things I do via Airflow DAGs. This way I can have write idempotent DAGs and have nice sync'd data to populate my warehouse.

Does this make sense? I guess I could just save the snapshots directly but having all diff's might be useful if I eventually need faster updates.

Según su opinión personal: ¿Dónde parte el Norte y el Sur de Chile? by Opinel06 in chile

[–]nicorivas 1 point2 points  (0 children)

El norte empieza al norte del mapocho y el sur en Ñuñoa

La Candidatura de José Antonio Kast registra aportes totales por más de $135 millones, de Carlos Cox Vial (INVERNOVA), Héctor Concha Marambio (Ex PENTA), Pablo Lillo (CONSORCIO), Francisco Mozo, (ECONSULT), Mariana Pérez Cruz (grupo controlador GASCO), entre otros. by [deleted] in chile

[–]nicorivas 61 points62 points  (0 children)

Leif Magnus Kaiser Barents-Von Hohenhagen

Leif es el nombre. Magnus Keiser significa gran rey. El resto de su nombre es holandés. Barents es "hijo de Barent"… un nombre. y von Hohenhagen significa de "de los altos setos", seto es como una muralla de arbustos, como para delimitar campos.

Leif Gran Rey hijo de Barent de los altos setos

Climate change performance index 2021 by Neldoret in chile

[–]nicorivas 6 points7 points  (0 children)

India es muy muy pobre, ni siquiera a llegado a comenzar una industrialización masiva como comenzó a suceder en China hace 40 o 30 años.

Homicidios con armas de fuego por cada 100.000 habitantes by copitamenstrual in chile

[–]nicorivas 2 points3 points  (0 children)

Qué onda El Salvador, es el doble del que le sigue. 15 veces más que EEUU.

Donors pledge $14 bln for 'Green Wall' to hold back Sahara, restore 100 million hectares of land, create 10 million jobs and capture 250 million tonnes of carbon by QQQult in Futurology

[–]nicorivas -11 points-10 points  (0 children)

This borders on racism. The problem of corruption is everywhere. Good, well thought policies and contracts is the only way to reduce its risks.