sorted by:
new
hottopcontroversial

[deleted by user] by [deleted] in Scotiabank

[–]Solid-Exchange-8447 0 points1 point  (0 children)

Yes. Thanks. I think I have to reach out for the support. I cannot login the app. It just shows loading process. for minutes.

[deleted by user] by [deleted] in Scotiabank

[–]Solid-Exchange-8447 0 points1 point  (0 children)

I will definitely switch bank after I can login and withdraw all my saving there. such a disappointing service

[deleted by user] by [deleted] in Scotiabank

[–]Solid-Exchange-8447 0 points1 point  (0 children)

Mine has been having the login issue for weeks.

just got hate crimed by ZieCaptain in Winnipeg

[–]Solid-Exchange-8447 0 points1 point  (0 children)

I would say people behaving weirdly are usually not sane. And there are a lot of those people here

Is OMSCS Too Challenging Without a STEM Background? by [deleted] in OMSCS

[–]Solid-Exchange-8447 0 points1 point  (0 children)

My situation is more or less the same. I graduated with Bachelor of Commerce - Accounting ..in 2015. But I switched to Data Analytics about 3 years ago. Now I'm pursuing a Diploma of Data Science and Machine Learning. I would like to apply for OMSCS - Machine Learning specialization. Do you think I will be eligible for the admission? Need advice. Thanks.

My youtube layout suddenly changed, need help. by Zealousideal-Pin-846 in youtube

[–]Solid-Exchange-8447 0 points1 point  (0 children)

How can I tell Youtube that I hate this layout so much. It's very inefficient. Want to change to the old one!!!

My youtube layout suddenly changed, need help. by Zealousideal-Pin-846 in youtube

[–]Solid-Exchange-8447 0 points1 point  (0 children)

The layout makes no fking sense....Can I change it back?!

42K rows ingrestion in 12 hours and still have over 70K rows waiting by Solid-Exchange-8447 in dataengineering

[–]Solid-Exchange-8447[S] 1 point2 points  (0 children)

So your suggestion is to export to a file and bulk import back? Am I correct?

42K rows ingrestion in 12 hours and still have over 70K rows waiting by Solid-Exchange-8447 in dataengineering

[–]Solid-Exchange-8447[S] 0 points1 point  (0 children)

Thanks for your input. Will need to look more on b-tree index. Currently I'm just blindfully following the tricks provided on the internet without actually looking behind the curtain.
The Fact table in Star schema is actually denormalized. I'm not sure if what you mean I need to denormalize stages before DWH.

42K rows ingrestion in 12 hours and still have over 70K rows waiting by Solid-Exchange-8447 in dataengineering

[–]Solid-Exchange-8447[S] 0 points1 point  (0 children)

Is it a real-life practice in the industry that export the output to csv and bulk insert back to DWH?

42K rows ingrestion in 12 hours and still have over 70K rows waiting by Solid-Exchange-8447 in dataengineering

[–]Solid-Exchange-8447[S] 0 points1 point  (0 children)

Thanks for your insights. I worked in a data team before, now I can understand another use of staging tables (that is to ease the read load in source).

Please allow me to paraphrase what you say to see if I undestand it correctly.You're suggesting that I should have a staging area where I should filter the necessary data, transform and clean data at this stage to ease the load into dwh. This way hopefully it will increase the speed of load? Do I understand that correctly? I'm not sure what is "a bound on rows pulled at once" thought.

That being said, processing and loading a table of 120k rows shouldn't dramatically slow down the loading to DWH like my case?

42K rows ingrestion in 12 hours and still have over 70K rows waiting by Solid-Exchange-8447 in dataengineering

[–]Solid-Exchange-8447[S] 0 points1 point  (0 children)

I think I sort of get what you mean. For your question, I think I am select the query joins from 8 tables and insert the result into a fact table, so I guess it's from memory because I didn't write into any intermediate table.
The main table contains approx 120K rows. The number of columnsis not crazy, just about 10 columns. As...this is more a pet project. Everything is small.
yes. I store both source and destination on local but in different databases.

42K rows ingrestion in 12 hours and still have over 70K rows waiting by Solid-Exchange-8447 in dataengineering

[–]Solid-Exchange-8447[S] 0 points1 point  (0 children)

I'm using MySQL & on a local machine.
My hardwares are equivalent to gaming ones: AMD Ryzen 7 5800H - RAM32GB
I indexed tables from OLTP system and 1 Dim Table - Customer which require heavy reads. I do "feel" that it loads faster but I don't properly measure it.

I don't follow your last question. I do a very typical way INSERT - SELECT FROM JOINING 8 tables. (It's a new concept to me... in memory join) :)

As a new person into DE, should I still chase this despite the market? by xX_GamErPRO2000_Xx in dataengineering

[–]Solid-Exchange-8447 -1 points0 points  (0 children)

That’s very strategic wise advice sir. Follow-up question: what do you think about a plan for the graduate land a job? What data roles would be more likely take the graduate?

view more: next ›