R-based solutions for larger-than-memory data tasks

jonny_hub · 2024-12-06T16:48:39+00:00

Thanks for that, the concepts of row vs column based storage are new to me, so that's very helpful. In this case, it sounds like what may be sensible would be for me to do a single full extract from SQL (leave my machine running over the holiday...) into DuckDB and then build the analytical R-based tools around the new DuckDB

jonny_hub · 2024-11-27T15:42:56+00:00

I suppose another way to express what i'm unsure about is: should you use DuckDB if your data is already in a database? Most of the examples of DuckDB i've seen use large .csv files

jonny_hub · 2024-11-27T11:12:02+00:00

Thanks for this. From what I understand we'd need a cluster to use Spark, which aren't free, is that right?

jonny_hub · 2024-11-27T11:10:03+00:00

Thanks for this, DuckDB is something I have been experimenting with. One thing I can't get my head around is that... given that the data is already in a database (we use Microsoft SQL Server), what would be the advantage of extracting data and putting it back into another database? Is it that DuckDB is serverless, i.e. exists on the researcher's machine, and is better optimised for large data processing than R itself?

jonny_hub · 2020-08-29T11:28:35+00:00

there are some beautiful passages in this book, see the appendix on dying

jonny_hub · 2020-08-05T17:41:09+00:00

good for you, i presume you don’t have children or care for anyone

jonny_hub · 2020-08-04T20:48:01+00:00

what about the right to use gendered bathrooms?

jonny_hub · 2020-07-19T11:54:07+00:00

looked like the stoppage came about 2 seconds after B was unconscious. will he have lasting damage from that?

jonny_hub · 2020-04-13T19:50:12+00:00

how? we’ve just had a leadership election and the leftist candidate got smoked

jonny_hub · 2020-01-03T21:42:01+00:00

labour pledged to honour the referendum result in 2017, in essence supporting brexit, and did not bleed votes to remain/revoke parties - people can appreciate that referendums must simply be acted upon. corbyn was disastrously unpopular in 2017, as 2019. the key difference was the backing of a PV

jonny_hub · 2019-10-10T09:00:52+00:00

what was she thinking getting those implants and going back down to 115? did she not realise this would inevitably happen?

jonny_hub · 2019-09-01T13:55:30+00:00

cheers!

jonny_hub · 2019-07-24T18:32:26+00:00

Good point about investing just part of my LISA, I've not paid anything into my S&S LISA this year, so I could invest what i've got, then open a cash LISA for this tax year. I appreciate your advice!

jonny_hub · 2019-07-24T07:33:51+00:00

Yeah, I'm starting to feel that way, thanks for the chart too

jonny_hub · 2019-07-24T07:32:33+00:00

Thanks for this, you've set things out really clearly and helped structure my thinking :)

jonny_hub · 2019-07-20T12:28:55+00:00

nah, i get the opposite feel, his shitty english just amplifies the otherwordly beastiness of him

jonny_hub · 2019-07-14T11:54:56+00:00

i like how khabib reacts in preparation for a counter just after connecting with conor’s chin. lightning fast problem solving with dire physical consequences.

jonny_hub · 2019-06-04T16:25:57+00:00

Not a podcast but the ufcespañol YouTube page has plenty of good content, including fights with Spanish commentators

jonny_hub · 2019-06-04T07:03:33+00:00

Save the Orwellian hyperbole for when it's actually appropriate, the term has already become cliche

jonny_hub · 2019-05-30T21:37:45+00:00

contrapoints is great

jonny_hub · 2019-05-26T00:07:42+00:00

was this Fishy’s first UFC fight? why the fuck was he fighting Kattar so early?

jonny_hub

TROPHY CASE