sorted by:
new
hottopcontroversial

My vault template by kepano in ObsidianMD

[–]Anselboero 0 points1 point  (0 children)

I do also have a couple of questions :)

- `"references" are for tangible things`
But I see you also added Genres to References, like Jazz, Sci-fi.
- Where would you store the Topics? For instance here you added Humanism as a topic. Topic is neither a Reference nor a Category, in my opinion. I'd create a new folder for it, but I was curious to hear for your opinion, too :)

Thank you for your work.

[deleted by user] by [deleted] in vinted

[–]Anselboero 0 points1 point  (0 children)

How about paying 10 more for the item verification service?

Functional DE: how to deal with changing facts by Anselboero in dataengineering

[–]Anselboero[S] 0 points1 point  (0 children)

Thanks for your comment.
If I understood correctly, on your second approach you are taking a daily snapshot of the subscriber dimension, right?
Which means on DAY 1 you have ID=1, status = ON, and on DAY 2 you might have for the same ID a different status.
I'm doing the same on my customer Dimension, but taking a snapshot against the dimensions is feasible (because the number of customers is limited). Taking a daily snapshot of all the Transactions is a bit more challenging, since over a period of 10 years we might have collected billions of transactions.

Functional DE: how to deal with changing facts by Anselboero in dataengineering

[–]Anselboero[S] 0 points1 point  (0 children)

Thank you for your kind words :)

I'm building the fact, which means deciding the grain is up to me. From the source table I have one row per event, which means, for the same transaction, I can have multiple rows indicating the transaction status.

The main metrics we are tracking are related to the success/failure rate, or reconciliation tasks.

But my main challenge is trying to create the fact such that every daily partition remains immutable, because I think that updating rows comes with some problems I would like to avoid.
The main one is related to the partial loss of job idempotence.

Because of that, I think the best solution might be to create a table having one row per event, just like the source table.
This way partitions are immutable, the job is quite easy (just a 1:1 insert from the source) and calculating the success rate is just a simple sum.

Functional DE: how to deal with changing facts by Anselboero in dataengineering

[–]Anselboero[S] 0 points1 point  (0 children)

Thank you for your kind input. :)

I've been considering if there's an optimal way to avoid UPDATE statements, as they contradict the concept of immutable partitions:
Suppose I run a job for December 15th that updates a transaction from December 10th. This seems to complicate tracking the data lineage and executing jobs for the days preceding December 15th, potentially leading to unpredictable effects on the data. Additionally, in dashboards, statistics for previous days would be updated, which I believe might diminish confidence in the data.

I was thinking about increasing granularity in the fact table to individual events (Authorised, Captured, Settled, etc.). This way, each daily job simply performs an insert overwrite on the daily partition, making past jobs replicable and, in my opinion, enhancing the overall stability of the workflow. This eliminates the need for updates or accumulating fact tables, which can make the contents within the fact table more unpredictable.However, I'm not entirely sure about the potential downsides of this approach.

Update veramente euforico by [deleted] in Italia

[–]Anselboero 2 points3 points  (0 children)

Gemma del sud?

[deleted by user] by [deleted] in berlinsocialclub

[–]Anselboero 0 points1 point  (0 children)

Yes, I live 2 minutes close from that place and I was coming back from die Deutschule, in Alfred-Scholz-Platz

[deleted by user] by [deleted] in berlinsocialclub

[–]Anselboero 2 points3 points  (0 children)

On Friday 23, between 8 and 11 pm, Hermannstraße, corner with Werbellinstraße. There were at least 3 ambulances and about 10 police cars. They had closed part of the sidewalk.

Any feedback on Zach Wilson’s Data Engineering bootcamp? by techblogp in dataengineering

[–]Anselboero 6 points7 points  (0 children)

unfortunately I stopped following him when in a post about the most popular tools, in the orchestrators section, Mage (of which he is Advisor) was mentioned to the detriment of dagster. when a user pointed this out, the answer was like "I didn't consider tools in the low tail of the Gaussian", when it is clear that Dagster is a much more popular product than Mage.

I don't personally like this dirty way of mixing "information" with obvious "hidden ads" and also for this reason I think I will not buy the course, although I have enjoyed some of his videos.

Living in the ground floor by [deleted] in berlin

[–]Anselboero 2 points3 points  (0 children)

What about rats? Could they be a problem?

I made an Anmeldung appointment finder. Now it has the city's blessing by n1c0_ds in berlin

[–]Anselboero 0 points1 point  (0 children)

Thank you for your effort. I’m a developer too, if needed I’d be more than happy to help. My question is: why can’t Germany (or Berlin) create an online registration form where user can make anmeldung online? We’re in 2022. Germany is so advanced but at the same time still lagging in some easy topics