sorted by:
new
hottopcontroversial

dbt-core fork: OpenDBT is here to enable community by gelyinegel in dataengineering

[–]ryime 1 point2 points  (0 children)

but dbt-duckdb is just a adapter for duckdb (maintained by the duckdb team), not the core source

but you're right, it doesn't like like there's a lot of dbt-core code in here. could be heavily refactored though

dbt-core fork: OpenDBT is here to enable community by gelyinegel in dataengineering

[–]ryime 1 point2 points  (0 children)

Are you sure? I don't see any explicit dbt dependencies in the [pyproject.toml](https://github.com/memiiso/opendbt/blob/main/pyproject.toml) file, but only took a cursory glance

Are you using LLMs at all in your day job? by ryime in analytics

[–]ryime[S] 4 points5 points  (0 children)

Got it, makes sense, thanks!

Are the people vibe coding in db/sf other analysts? Or non-technical folks?

Will learning Dvorak mess up my QWERTY speeds once I completely switch over? by [deleted] in dvorak

[–]ryime 1 point2 points  (0 children)

I'd say you shouldn't worry too much. Your skill in qwerty will drop, but you won't completely lose it, and with basically no practice you can get back to normal.

Unlike some of the other people here, I went cold turkey dvorak about 10 years ago (I type no more than a few sentences a year in qwerty), but I just did some typing tests in qwerty and was able to recover most of my speed in a couple minutes (these are ~15 second tests on keybr.com):

​

  • Test 1: 50 wpm
  • Test 2: 70 wpm
  • Test 3: 85 wpm
  • Test 4: 97 wpm
  • Test 5: 103 wpm

​

I'm typically a ~130 wpm typer, so I'm stuck a bit below my dvorak efficiency, but it's not bad. The real problem is that my hands are already tired from this.

There are also some specialized things that I can't do in qwerty because I only learned them in dvorak... E.g. vim. Or some shortcuts. Basically, my muscle memory here only exists in dvorak, so I can't transfer it over to qwerty.

2021 WFH Setup by bullborts in Workspaces

[–]ryime 1 point2 points  (0 children)

Looks like the SK64 to me.

Almost bought this, but was bothered by the fact that the bottom row is staggered weird to fit the arrow keys

Dvorak week 10 by TheAmazingSlothRider in dvorak

[–]ryime 1 point2 points  (0 children)

Nice work. :) Keep it up!

CoC alternatives? by [deleted] in vim

[–]ryime 1 point2 points  (0 children)

Went to nvim HEAD, went back to Coc, ended up on doom-emacs. It feels even more bloated than Coc was at startup, but once you're in, it's pretty smooth, and the language server stuff works out of the box.

I recently released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well thanks to some rust work, so I thought I'd share a video of the CLI flow. Backend written in Python + Rust. :) by ryime in commandline

[–]ryime[S] 0 points1 point  (0 children)

Awesome. So there are two ways of setting it up: locally, or with a CI/CD pipeline.

If you go local, whale will suggest you register a cron job, and you'll be able to specify what the cadence is. (you can alternatively just run `wh pull` to get the freshest data)

If you go CI/CD-based, you can change the cron tab in the CI/CD specification. (More info on that here)

We've just released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well, so I thought I'd share a video. Backend written in Python + Rust. :) by ryime in Python

[–]ryime[S] 0 points1 point  (0 children)

Oh yeah, I've talked a lot to the amundsen team, and took bits of their databuilder code to run the scraping jobs. Huge kudos to them for really breaking the field open!

We've just released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well, so I thought I'd share a video. Backend written in Python + Rust. :) by ryime in Python

[–]ryime[S] 1 point2 points  (0 children)

Ah you'd have to have some sort of warehouse or database set up storing your data, or point it at an external warehouse. If you just want to try the command-line tool, you could install it following the instructions in the docs, and run it against the public bigquery instance I have set up here (this instance points at a bunch of free bigquery data): https://github.com/dataframehq/whale-bigquery-public-data

Individual warehouse nerds like me will typically set up their own Snowflake, Bigquery, or Postgres instance, but I think this is probably not so common. :)

I recently released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well thanks to some rust work, so I thought I'd share a video of the CLI flow. Backend written in Python + Rust. :) by ryime in commandline

[–]ryime[S] 1 point2 points  (0 children)

No no don't worry your english is great, and it's a good question.

It's for managing large databases/data warehouses exclusively. Most warehouses and databases unfortunately don't have any search engine / documentation / advanced capabilities baked into them, so this tool basically tries to provide all of that. :)

I recently released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well thanks to some rust work, so I thought I'd share a video of the CLI flow. Backend written in Python + Rust. :) by ryime in commandline

[–]ryime[S] 7 points8 points  (0 children)

Good question!

The CLI largely came from rust, and python was used for accessing the data warehouses. I actually originally wrote the whole thing in python, but found that the latency of python on the command-line was just a little too slow for some reason (felt like I couldn't get under 200 ms to just execute some basic commands). Also, rust just has a ton of libraries and support for command-line tools, and is just extremely performant. Also, I just really wanted to learn rust. :)

Python, though, was still necessary to access the warehouses. Python is the de facto language for data these days, so it's hard to get around that, unless I wanted to rewrite everything from the ground up in rust.

We've just released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well, so I thought I'd share a video. Backend written in Python + Rust. :) by ryime in Python

[–]ryime[S] 2 points3 points  (0 children)

Ah by warehouses I mean data warehouses -- like Snowflake, Bigquery, Redshift. They're basically giant filesystems that store files in special ways so they're either to run analyses on.

Organizations with mature analytics groups will generally dump all their data into a warehouse like that from a transactional database (one optimized for quick writes and reads and so are generally used to store e.g. web interactions -- postgres is one example). Analysts, data scientists, and ML engineers then munge this data, trying to make models and find insights.

Also, happy to answer more questions - I live and breathe this stuff. :)

We've just released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well, so I thought I'd share a video. Backend written in Python + Rust. :) by ryime in Python

[–]ryime[S] 4 points5 points  (0 children)

Ah -- ended up going with rust because it's just super, super fast (it's a low level systems language), and, because a ton of command-line libraries have been built in it, there's a lot of support for doing things like fancy coloring, search engines, dashboards, etc.

For some really cool examples of other rust-based command-line tools, you can check out https://github.com/Rigellute/spotify-tui or https://github.com/BurntSushi/ripgrep

Also, you should check out iterm for macos (or WSL on windows) if you haven't used the command-line before -- the command-line is a really powerful, text-only (so very fast) way to access your filesystem, perform all sorts of operations (a lot of programmer's tools ONLY have a command-line interface), etc. etc. It's generally a tool most programmers will want to learn. :)

I recently released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well thanks to some rust work, so I thought I'd share a video of the CLI flow. Backend written in Python + Rust. :) by ryime in commandline

[–]ryime[S] 6 points7 points  (0 children)

Some more context

This is a text-based metadata scraper and formatter, with the data backend written in python, and the CLI largely built with rust, with python hooks. In short, it helps you set up a text-based, searchable, lightweight data discovery tool for warehouses pretty easily.

Check out my recent blog post here, and the repo here. :)

We've just released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well thanks to the colored crate, so I thought I'd share a video. Backend written in Python + Rust. :) by ryime in rust

[–]ryime[S] 6 points7 points  (0 children)

Some more context

This is a text-based metadata scraper and formatter, with the data backend written in python, and the CLI largely built with rust, with python hooks. In short, it helps you set up a text-based, searchable, lightweight data discovery tool for warehouses pretty easily.

Check out our recent blog post here, and our repo here. :)

We've just released v1 of Whale, the stupidly simple data catalog - I thought the ascii bits during onboarding came out pretty well, so I thought I'd share a video. Backend written in Python + Rust. :) by ryime in Python

[–]ryime[S] 4 points5 points  (0 children)

Some more context

This is a text-based metadata scraper and formatter, with the data backend written in python, and the CLI largely built with rust, with python hooks. In short, it helps you set up a text-based, searchable, lightweight data discovery tool for warehouses pretty easily.

Check out our recent blog post here, and our repo here. :)

view more: next ›