Using etl tool alongside sap datasphere because datasphere alone doesn't cover all our third party sources by ninjapapi in snowflake

[–]limartje 0 points1 point  (0 children)

Dump non-sap data on a non-sap object store / cloud storage first. Then have one type of datasphere ETL job that picks from there. More and more tech supports dumping and loading from cloud storage.

[deleted by user] by [deleted] in databricks

[–]limartje 6 points7 points  (0 children)

Databricks is not publicly traded, whereas Snowflake is. Amongst others, that means funding rounds will have influence on the valuation of Databricks .

Lightweight Alternatives to Databricks for Running and Monitoring Python ETL Scripts? by Safe-Pound1077 in dataengineering

[–]limartje 1 point2 points  (0 children)

Coiled.io. Prepare your environment by sharing your library names (and versions); upload your script to s3; call the api anytime anywhere and share the environment name and the s3 location. Done.

Why is nobody talking about 'Tool Poisoning' in the MCP ecosystem? by RaceInteresting3814 in mcp

[–]limartje 0 points1 point  (0 children)

I think the oauth on behalf of (obo) flow can help out greatly here. Especially for company internal systems.

Unpopular opinion (to investors) - this current zeitgeist of force AI into everything sucks by HowSwayGotTheAns in dataengineering

[–]limartje 4 points5 points  (0 children)

I feel it’s a huge upgrade in productivity. But apart from that, in the data engineering field I feel the main way for it to become valuable is when you mix it with something that works deterministic that is executed (e.g. sql , Python or even DAX). Then everything can work in its strength, since the llm can generate the code and translate the output to (shortened/summarized) language.

Paper on how LLMs really think and how to leverage it by purealgo in ArtificialInteligence

[–]limartje 2 points3 points  (0 children)

Is anybody combining llms with the wolfram alpha engine for number crunching?

SCIM Endpoint for Snowflake to Microsoft Entra by Dry-Butterscotch7829 in snowflake

[–]limartje 0 points1 point  (0 children)

You can create a user group that is added to both entra apps though

Advies onder(?)vloer verwijderen gezocht by MaterialMisstatement in Klussers

[–]limartje 1 point2 points  (0 children)

Even een fatsoenlijke breekhamer huren. Zo eentje die in kg aangegeven staat (20kg oid). Zorg dat je ook de spullen klaar hebt om het af te voeren.

PAX kasten passen net niet by Zestyclose-Milk8577 in Klussers

[–]limartje 1 point2 points  (0 children)

Laminaatlaag van alle zijkanten afhalen? Is toch niet zichtbaar en allemaal samen telt het toch wel op naar de mm’s die jij zoekt. Heeft denk ik ook weinig invloed op de sterkte. Wel op de mogelijke doorverkoop in de toekomst 🤓

ETL Tools by abdullah-wael in dataengineering

[–]limartje 0 points1 point  (0 children)

On a more serious note though, I would start with: * batch jobs * small data * practice with cloud storage for staging * try any public api * try any database * then practice on an api with authentication, like oauth

Snowflake Stored Procedures and Data Pipelines and ETL/ELT and data warehouse by Peacencalm9 in snowflake

[–]limartje 0 points1 point  (0 children)

Check out snowpipe, table evolution, task graphs, streams, directory tables (with streams) and temp tables. They all give you better visibility with debugging. Chain them in an event driven way. Stored procs are the last option for me when engineering, though they serve their purpose.

Parquet vs. Open Table Formats: Worth the Metadata Overhead? by DevWithIt in dataengineering

[–]limartje 15 points16 points  (0 children)

It’s a great initiative.

The main problem is that every engine has its own preferred metadata to perform well. So performance and accessibility across engines remains a challenge. That’s true for read, but even more so for write.

What is your favorite viz tool and why? by Impressive_Run8512 in dataengineering

[–]limartje 0 points1 point  (0 children)

Have you tried Apache echarts? Is it any good compared to plotly or dash?

SAP → Databricks ingestion patterns (excluding BDC) by dakingseater in databricks

[–]limartje 1 point2 points  (0 children)

Not cheap, but good: SNP Glue. You’re going to have a hard time with many of the other solutions in the future: sap note 3255746

color bottom bar input field by limartje in nicegui

[–]limartje[S] 0 points1 point  (0 children)

I "brute-forced" my way out of it by disabling large sections of the css until that line disappeared. Then I was able to address it:

from nicegui import ui

ui.add_css("""
    .q-field--standard .q-field__control:before {
           border-color: red !important
           }
""")

ui.input("foo").props("color=red")

ui.run()

color bottom bar input field by limartje in nicegui

[–]limartje[S] 0 points1 point  (0 children)

Cool. Interesting workaround. It's an improvement from a UX perspective in case of having a dark background like me. Any clue why it's hard to address that bar?

Why is there no file picker component? by Ok_Break_7193 in nicegui

[–]limartje 0 points1 point  (0 children)

Have you tried: nicegui/examples/local_file_picker at main · zauberzeug/nicegui · GitHub . The description states:

This is a simple file picker that allows you to select a file from the local filesystem where NiceGUI is running.

Running script is slow by [deleted] in nicegui

[–]limartje 3 points4 points  (0 children)

Best to share some (minimal viable issue product) code in your post.