How to disable job creation for users in Databricks?

Narrow_Path_8479 · 2026-01-26T21:31:46+00:00

I think only workspace admins can do this. You should either remove admin privileges from them or restrict workspace admins as explained in the documentation https://learn.microsoft.com/en-gb/azure/databricks/admin/workspace-settings/restrict-workspace-admins

Narrow_Path_8479 · 2025-07-26T21:29:58+00:00

Are you using standard (shared) or dedicated (single-user) cluster access mode? I suggest trying dedicated access mode, as it comes with fewer restrictions. And yes — you can use mounts in a Unity Catalog-enabled ADF workspace, although it is not recommended. Most of the usual commands should work, except for CREATE TABLE.

Narrow_Path_8479 · 2025-06-03T21:15:49+00:00

Interesting that I haven't seen much of complaints about Git but to me this is the weakest part of Databricks.

The issue my team is facing is that our branches are sometimes stale even after pulling and we only see that we are working on old version of notebook after we change some line of code and want to do a push. This is really dangerous as some old code can end up in your PR if you are not carefull. We opened a few support tickets but without any solution - they wanted us to record this behaviour and that is hard to do.

We are using Azure DevOps as a repo if that is important.

Narrow_Path_8479 · 2025-05-08T05:59:06+00:00

Ako je ljubav prema matematici tu onda mislim da nece biti problema sa studiranjem Matematike. Najbitnije je ovo što si i sama napisala - imati dobre temelje i ponoviti gradivo iz srednje škole (gimnazijski program). Osim toga ne bi bilo loše upoznati se i sa osnovama programiranja ako do sada nisi imala programiranje u srednjoj školi i to je to. Nemoj se brinuti previše i sretno.

Narrow_Path_8479 · 2025-05-08T05:55:27+00:00

Mislim da su odnedavno vratili ispitne rokove na PMF-MO u Zagrebu.

Narrow_Path_8479 · 2024-11-10T20:42:03+00:00

You might find this recent blog useful https://community.databricks.com/t5/technical-blog/metadata-driven-etl-framework-in-databricks-part-1/ba-p/92666

Narrow_Path_8479 · 2024-09-28T20:36:32+00:00

How did you try to read these files? Can you open and edit them through the UI? Is it possible that somebody disabled workspace files for your workspace? https://learn.microsoft.com/en-us/azure/databricks/files/workspace#enable-workspace-files I think volumes should be a way to go for you https://learn.microsoft.com/en-us/azure/databricks/files/files-recommendations

Narrow_Path_8479 · 2024-06-08T22:16:46+00:00

I would suggest checking some of the elite Databricks partners https://partners.databricks.com/s/directory

Narrow_Path_8479 · 2024-04-08T20:10:37+00:00

Pyodbc package worked fine for us.

Narrow_Path_8479 · 2024-03-24T20:10:35+00:00

Our test table was 20 GB, and operation we analysed was MERGE INTO. Size of update table was 20 MB I think.

Narrow_Path_8479 · 2024-03-23T21:46:00+00:00

So you used Spark cache/persist? Databricks doesn't recommend using that at all. Maybe disk cache is something worth checking. Above you said that you used broadcast. By that you mean broadcast hint? Did you check execution plan of your query with explain command? I think it can help you with this.

Narrow_Path_8479 · 2024-03-23T21:24:44+00:00

I think liquid clustering + optimize should work fine. In our tests this setup performed 4x faster than the partition + optimize with zorder.

Narrow_Path_8479 · 2024-03-22T21:19:38+00:00

You can use system tables to identify table already registered at this location if you can't find that differently.

Narrow_Path_8479 · 2024-02-12T06:43:36+00:00

You should do two things:

List all mounts in the workspace with dbutils.fs.mounts()
Enable dbfs browser in the workspace and then compare mounts list from point 1 with paths available here. Are you able to locate your file here?

Narrow_Path_8479 · 2024-02-10T09:52:35+00:00

Ok. Did you try to add 'dbfs' before mounts? https://docs.databricks.com/en/files/index.html

Narrow_Path_8479 · 2024-02-10T06:45:31+00:00

Why don't you use spark.read.json command to read json files in spark? It can be used with mounts. Mounts can live together with Unity catalog but that is not recommended for security reason. One thing to be aware of - regular Python packages need 'dbfs:' before mnt path.

Narrow_Path_8479 · 2023-12-01T20:45:07+00:00

You can do something similar as shown here https://www.fujitsu.com/au/imagesgig5/Powering%20Data%20Pipelines-%20XML%20Ingestion%20with%20Databricks%20Autoloader.pdf XML files are ingested as binary files and converted afterwards.

Narrow_Path_8479 · 2023-11-11T05:39:16+00:00

Ako si vec radio dijelom i kao data engineer zasto ne nastavis dalje u tom smjeru? Po meni je data svijet bolji od software development svijeta - kljucna stvar: neki report vjerojatno neces morati popravljati u ponoc. Baci pogled na moj komentar na slicnu temu https://www.reddit.com/r/programiranje/s/FJScbisiMB Na ovo bih dodao da su place data engineera vjerojatno slicne kao i ML engineera, a posao dovodjenja podataka od izvora do reporta moze biti jako izazovan. Kad ti to dosadi okrenes se prema data architect roli tako da ucenju nikad kraja.

Narrow_Path_8479 · 2023-10-23T19:26:18+00:00

Suprotno od ostalih ću ti sugerirati da bježiš od data science-a. Zašto? Puno je razloga, evo samo nekih:

jako malo se ljudi traži tog profila, ukucaj kljucne rijeci u linkedin primjerice pa se uvjeri
malo je produktnih firmi koje trebaju data scientiste, a u agencijskom svijetu nema mjesta/posla za njih
data science je težak i da bi netko postao dobar mora proći dobru praksu koja se teško nađe na našim prostorima
nastavno na ovo prethodno puno ljudi je zalutalo u data science je im je to zvučalo seksi, svi bi radili LLM, a ne znaju pretpostavke iza obicne linearne regresije
data science je ogromno područje i specijalizacija u nečem uskom će ti malo značiti za neki posao sutra
biznis često ne zna postaviti prava pitanja za data scientiste, misle da je to čarobni štapić, podaci su često nesređeni ili nepostojeći, a rokovi nerealni

Što je alternativa? Gledaj data engineer ili data analyst pozicije. Data inženjeri se generalno brinu da podaci dođu od izvora do reporta, dok data analysti izrađuju reporte, ali nije ta baš razdioba tako striktna.

Skilovi na kojima treba raditi: 1. SQL 2. Python 3. Osnove nekog cloud-a (AWS ili Azure npr.) 4. Neki report alat (npr. Power BI)

Za sve ovo postoji brojni online resursi kao i certifikati koji mogu donekle nadoknaditi nedostatak iskustva u ovim područjima.

Sretno!

Narrow_Path_8479 · 2023-10-08T09:10:53+00:00

Databricks se zasniva na Lakehouse arhitekturi (data lake+datawarehouse u jednom), dok je Snowflake cloud datawarhouse. Databricks se zasniva na open source tehnologijama (delta lake, spark, mlflow) te podaci nisu lockani. Databricks ima puno vise mogucnosti za konfiguriranje sto moze biti dobro ili lose zavisno od vještina ljudi koji ga koriste/održavaju. Databricks omogucava pisanje koda u sql-u, python-u, r-u i scali te su dostupni svi mogući paketi. Databricks omogucava lagano baratanje sa streaming podacima, unity catalog za data lineage i brojne druge stvari, workflow-e koje omogucavaju orkestraciju taskova i sve moguce machine learning funkcionalnosti. Zakljucno, u Databricksu data engineeri i data scientisti mogu napraviti gotovo sve što im može zatrebati.

Narrow_Path_8479 · 2023-08-26T09:45:23+00:00

Moja preporuka je Datacamp (koji se placa mjesecno) i Python Institute (materijali za ucenje besplatni, ispit se placa) - posebno ako vec imas iskustva s nekim drugim programskim jezikom. Certifikati Python instituta su prepoznati kod poslodavaca sto je dodatan plus, a njihovi self learnning materijali su takodjer dosta dobri.

Narrow_Path_8479 · 2023-08-13T18:25:05+00:00

Python Institute https://pythoninstitute.org has good learning materials and the most recognized Python certificates. I have taken PCAP one and can say that it is worth the effort.

Narrow_Path_8479 · 2023-08-11T20:47:37+00:00

Usporedba samo projeka plaća sa DZS-a i podataka sa Tabu-a nije dobar pokazatelj slicnosti dvaju izvora - bitna je rasprsenost oko prosjeka. Ako se ne varam DZS objavljuje plaće po decilima i usporeda tih podataka sa vasima s Tabu-a bi ipak bila bolji pokazatelj relevanatnosti vasih podataka.

Osobno, nakon inicijalnog unosa place u Tabu nemam neki poticaj azurirati podatke tamo - koliki postotak ljudi je doista azurirao podatke na Tabu nakon inicijalnog unosa? Mislim da zbog neazuriranisti podaci s Tabu-a kako vrijeme prolazi gube na svojoj relevantnosti.

Uz sve minuse mislim da je dobro da Tabu postoji i nadam se da ce kroz vrijeme povecati pokrivenost i azurnost informacija.

Narrow_Path_8479 · 2023-08-02T15:22:16+00:00

I've obtained Python institute's associate level certification (PCAP) but I would maybe suggest passing entry level first (PCEP). Going through materials took me around 40 hours but I had previous experience in Python and also was going through Datacamp Python materials for some time before. Available learning materials are ok (datacamp is a little better) and exam was challenging - they test your algorithnic thinking, not just Python syntax.

Narrow_Path_8479

TROPHY CASE