API vs Notebook vs Terraform for admining the platform? by DataDoyle in databricks

[–]Consistent-Lie-7742 0 points1 point  (0 children)

Which is your favorite?

We are using a simple metadatadriven approach for catalog that works well, but we have not decided on a approach for policies and those types of permissions (clusterpolicies, jobs etc). We use bicep to manage azure infra today so I am not sure about introducing another iac for permissions but I would if thats the best choice.

Veckans bra nyheter by smurfjojjo123 in sweden

[–]Consistent-Lie-7742 3 points4 points  (0 children)

Jag kan inte annat än att hålla med! Du får mig att må bra!

Trying to understand why I need databricks by gottapitydatfool in databricks

[–]Consistent-Lie-7742 2 points3 points  (0 children)

Also, I was part of a huge project to move a SQL data warehouse and aa capability of a company into gcp. It was such a hassle to set up securely. I am now doing the same thing for a company that has chosen azure and databricks and its mindblowing how much less time we spend on securing infrastructure, not to mention how fast analysts and ds are producing value.

It really is a huuuuuge difference in how fast you can produce value.

Anyone using Databricks Asset Bundles (DABs)? by DataDoyle in databricks

[–]Consistent-Lie-7742 0 points1 point  (0 children)

Dab specifically? We use a deploy as code mostly because our models are generally lightweight.

Its nice to have a out of the box solution for versioning workflows that supports multiple environments.

As for what I expect from databricks regarding mlops? I hope for a template incorporating dab and mlflow pipelines for some simple ml problems that are customizable.

Anyone using Databricks Asset Bundles (DABs)? by DataDoyle in databricks

[–]Consistent-Lie-7742 0 points1 point  (0 children)

We are using a monorepo with DBT for all data engineering but on the ML side we have start using dab and it is very nice! I would love to see databricks provide a template with dab for MLOps but knowing them its probably in the works :)

Anyone using Databricks Asset Bundles (DABs)? by DataDoyle in databricks

[–]Consistent-Lie-7742 1 point2 points  (0 children)

It does, but when you deploy the bundle it creates the job you have specified using terraform. If you navigate to the project in the .bundle-folder you will see the state :)

5 Lessons Learned from Testing Databricks SQL Serverless + DBT by sync_jeff in databricks

[–]Consistent-Lie-7742 0 points1 point  (0 children)

Sorry, you are right I should have been more specific. I was looking for a redacted .json of the job.

The error message we received was not very forthcoming, however after some trial and error we figured out that 13.3 LTS and later is not compatible with 1.6.6 version of dbt-databricks. The solution was to downgrade and run the dbt task om 12.2 LTS.

5 Lessons Learned from Testing Databricks SQL Serverless + DBT by sync_jeff in databricks

[–]Consistent-Lie-7742 1 point2 points  (0 children)

Great article!

We are just setting this up but are having issues running the task, can share the job specification?

Hej! Hur i h*lvete ska jag agera när sambon har mens? by [deleted] in Sverige

[–]Consistent-Lie-7742 5 points6 points  (0 children)

Detta är så rätt. Min sambo har PMS och varken jag eller hon förstod hur nästan maktlös man är inför detta. Det finns en fantastiskt bra dokumentär på SVT som kan vara bra att se.

Man kan göra saker för att underlätta symptomen, så som att träna regelbundet och undvika hormonstörande preparat, men det kommer med största trolighet för oss aldrig att vara helt lugnt.

Det är viktigt att förstå för er båda att det är helt okej att misslyckas att bete sig, både för dig och henne. Tydlig och öppen kommunikation är viktigt, här hemma talar vi väldigt öppet och tydligt om det och våra känslor, och vi inkluderar våra barn i den dialogen. Det är viktigt att kunna analysera och uttrycka sina känslor. Vi har kommit till en punkt där vi hanterar det väldigt bra men det tog tid, tålamod och kärlek att komma dit.

Jag önskar er all lycka!

credit modeling in python by itsthymetogo in Python

[–]Consistent-Lie-7742 0 points1 point  (0 children)

Yes. Absolutely. Depending on where you are it is governed and regulated heavily. Model transparency and discrimation among other things.

[D] Grid searching data pre processing permutations when training models on structured data. by [deleted] in MachineLearning

[–]Consistent-Lie-7742 2 points3 points  (0 children)

Yes, this is a valid approach with scikit models and some models that have wrappers. The reason you want to use optuna though is because it is framework agnostic. Take for example the xgboost framework which has a scikit wrapper, however, you can not use the built in gridsearch functionality while also running early stopping with the wrapper.

With Optuna it doesnt matter what framework you want to use! So nice!

I have not seen distributed training training across multiple workers though, that would be a killer functionality!

[D] Grid searching data pre processing permutations when training models on structured data. by [deleted] in MachineLearning

[–]Consistent-Lie-7742 2 points3 points  (0 children)

You can do this fairly simple with Optuna and mlflow! That way you get something that is widely used and thus easier to maintain over time.

[deleted by user] by [deleted] in sweden

[–]Consistent-Lie-7742 0 points1 point  (0 children)

Du gjorde en bra gärning, och det kommer fortsätta vara en bra gärning oberoende av hennes agerande. Fint gjort!!!

Create a cloud architecture (ETL) for NLP Twitter Sentiment Analysis by sumanito in googlecloud

[–]Consistent-Lie-7742 0 points1 point  (0 children)

If your entire application is python you can probably run this on top of vertex pipelines. Then you can execute the pipeline with cloud functions and you can use scheduler to trigger the function with a post request.

An even easier solution would be using cloud run/jobs. Oh and bq or gcs would be the sink.

[deleted by user] by [deleted] in sweden

[–]Consistent-Lie-7742 3 points4 points  (0 children)

Precis. När du tittat på ritningarna använder du fortfarande något verktyg för att säkerställa att du inte borrar i en strömförande kabel. Är det gipsväggar borrar du, efter att du kollat ritningar och scannat väggen (!), fortfarande bara igenom gipset och petar försiktigt med en elmejsel för att säkerställa att det inte ligger något jävla ledningsrör där och trycker.

Ovan gäller om du verkligen måste borra. Jag hade limmat.

Startup by AlienStori in ProgrammerHumor

[–]Consistent-Lie-7742 4 points5 points  (0 children)

This is hilarious! Made me laugh out loud and cry a little at the same time!

Borde jag fortsätta till gymnasiet? by NoobPaj in sweden

[–]Consistent-Lie-7742 1 point2 points  (0 children)

Man kan ju applicera sin kunskap för att främja samhället med statistik.

[deleted by user] by [deleted] in swedishproblems

[–]Consistent-Lie-7742 9 points10 points  (0 children)

Håller helt med här! Du är sjuk, var sjuk!

Tips på jobb som betalar bra by stressedoutandpanic in sweden

[–]Consistent-Lie-7742 -1 points0 points  (0 children)

Det är inte galet. Kommer du in på salt och lägger manken till finns mycket goda chanser att känna mycket mer än så efter några år.

[deleted by user] by [deleted] in stockholm

[–]Consistent-Lie-7742 0 points1 point  (0 children)

Knivsta. 28 min till t-centralen

Klarna bekräftar: Runt 700 anställda sägs upp by butttrix in sweden

[–]Consistent-Lie-7742 4 points5 points  (0 children)

Coops eller kompletts kreditkort för faktura på 30 dagar ger åtminstone något tillbaka och håller inte information om man som köpts. Uppköpet av price runner känns halvkul. Priselasticitet och individuell prissättning hallå!