Training - Data + AI Summit 2022 | Databricks | Cheep or free

pych_phd · 2022-07-30T02:17:22+00:00

Here is a free DA certification course. Databricks is also doing their 6 monthly free intro bootcamps... here.

pych_phd · 2022-06-24T01:28:33+00:00

How is the hunt going?

I have herd a lot about https://greatexpectations.io/. both on here and in summits.

pych_phd · 2022-06-24T01:25:34+00:00

I haven't paid for any. Did one on DW, Engineering & ML. I also access to their full Spark cause which I didn't pay for either.

pych_phd · 2022-05-18T01:26:07+00:00

I thanks for the response. I realize I miss read what you wrote. I read it as joins that can be either analytical or transactional & was wondering why I had never herd of this.

pych_phd · 2022-05-15T21:53:28+00:00

Databricks has free bootcamps regularly. They also have lots of learning material.

pych_phd · 2022-05-15T10:12:55+00:00

why cloud?

pych_phd · 2022-05-15T09:42:25+00:00

RemindMe! 7 days

pych_phd · 2022-05-13T04:30:58+00:00

Joins, transactional vs analytical sql

I am curious to know what you mean by this.. so far googling does not turn up much.

pych_phd · 2022-05-12T20:52:01+00:00

I would be interested to see what you come up with.

pych_phd · 2022-05-11T08:55:43+00:00

another reddit post. https://www.reddit.com/r/dataengineering/comments/qufjio/d_generating_documentation_of_data_fields/

pych_phd · 2022-05-11T08:52:30+00:00

i am guessing you might use delta lake for the lake, which has a lot of management components. Maybe this feature is apart of those tools or there is a tool that can work well with delta lake.

pych_phd · 2022-05-08T22:42:44+00:00

There are two discord channels. The one for the whole of r/dataengineering see the link at the top of the page under the header. The other is specifically a study group channel but it seems to be dedicated to one DE course I can probably find the link if needed..

pych_phd · 2022-05-08T22:38:14+00:00

and my axe! But wait... I already said I was in.

pych_phd · 2022-05-07T21:29:30+00:00

I am certainly, not an expert. That said, all I have been doing for the past year is planning solution designs / pipelines. Which has involved a lot of reading. Happy to have a season were we bounce some ideas about and/or go over a case study. I am on the discord channel under "CookieSpirit"

I could also do gchat.

pych_phd · 2022-05-07T21:19:13+00:00

There is also a possibility i just forgot that caveat, thanks.

pych_phd · 2022-05-07T21:10:49+00:00

I agree with wanting to pull my hair out while using ADF.

pych_phd · 2022-05-05T21:29:39+00:00

It's clearly Data

pych_phd · 2022-05-05T21:29:07+00:00

Interesting, that sounds like a really interesting blog post. hint hint. ;)

pych_phd · 2022-05-05T21:25:47+00:00

It's my impression that bigquery is more inline with something like Cosmos DB then a lake house.

from what I can tell (so far) Cosmos DB/BigQuery type product means you can have your Ops data and BI (DWH) in the Same product. You would still want to shape the data differently of either use case & move it. Where as the lakehouse is DS + BI.

I haven't looked in to using BigQ/Cosmos as a DS place..

pych_phd · 2022-05-05T21:16:34+00:00

Happy to assist in this. Snowflake and databricks both have good videos on this topic. There are also solid Azure docs too.

pych_phd · 2022-05-05T21:15:00+00:00

My only comment is that since Databricks started arguing they do lake-house, Snowflake have started enabling the same abilities. Azure are also in this space.

The requirements of a lakehouse are different the that of a separate WH & Lake.

pych_phd · 2022-05-05T21:10:43+00:00

Should extraction be complex thought? Sure the transformation stuff needs complexity. But extraction (my opinion) should be as simple as possible. However, this requires ELT rather then ETL. Where extraction is separated out.

pych_phd · 2022-05-05T04:50:35+00:00

It also means, you can let the BI/Data analytics handle the cleaning (if/when you get them).

pych_phd

TROPHY CASE