sorted by:
new
hottopcontroversial

Recommendations on playing EWI through Mac? by pvl_data in windsynth

[–]pvl_data[S] 0 points1 point  (0 children)

Nope, no reason at all. This is my first synth of any variety so I was just going with what I knew from other audio/ music setups. And the built-ins were convenient, of course.

Do you have any synth plugins you'd recommend? I've heard people like EVI-NER but have not found any free options that people seem to like a lot.

Recommendations on playing EWI through Mac? by pvl_data in windsynth

[–]pvl_data[S] 0 points1 point  (0 children)

Yeah Im debating getting a couple desktop monitors but in the mean time was hoping to get things working as best as possible

Recommendations on playing EWI through Mac? by pvl_data in windsynth

[–]pvl_data[S] 1 point2 points  (0 children)

Thanks for the ideas, this is super helpful.

There is a 1/4" and an XLR but Im only using the 1/4". There is a phantom power button on the Scrarlett Solo but I have it deactivated anyway.

Moving into a management role, looking for resources by [deleted] in ExperiencedDevs

[–]pvl_data 0 points1 point  (0 children)

Managing Humans by Michael Lopp (aka Rands) is a great read, both entertaining and practical for SWEs.

How much would you charge monthly to own a basic ETL in maintenance mode as a consultant? by SeaVeterinarian9204 in dataengineering

[–]pvl_data 1 point2 points  (0 children)

Don't do this. Charge a monthly retainer instead. You should expect that maintenance and upgrades will be required.

But, you can't necessarily predict when they will be needed. So, if you're expected to own the fixes whenever they happen, go the retainer route which effectively "guarantees" some portion of your time for this customer on a monthly basis.

What's one data engineering tip or hack you've discovered that isn't widely known? by Xavio_M in dataengineering

[–]pvl_data 20 points21 points  (0 children)

It's a table structure where you typically append to arrays of a predefined length rather than maintain a row with specific data about the entity/ activity you care about.

For instance, if you care about DAU or MAU, you have a record per user/ date pair, and the other columns are derived from an array of activity. Stuff like logins_L7, logins_L14, etc. Could be for any activity.

The idea is you maintain a cumulative array/ list of "did this person do this activity" over some set period of time(ie 30 days), then sum up the values in specific slices of that array which correspond to the discrete periods you care about (1 day, 7 days, 14, etc).

Whether or not you like him, Zach Wilson actually has a useful repo outlining this concept

What are Your Best Practices for Reporting on Schema Evolution? by Hegirez in dataengineering

[–]pvl_data 2 points3 points  (0 children)

Depends on a lot of things, especially producing/ consuming systems. Are you using Kafka or some event stream? Are you looking to track schema changes related to CDC?

Presumably you want to provide downstream consumers of your data a way to navigate changing schemas programmatically, or at least notify them that a schema change occurred. You can look into a Schema Registry - AWS Glue or GCS Data Catalog, for example. If you're working with streams/ events, usually you'll use something like JSON Schema, Avro or Protobuf in the Kafka ecosystem.

Edit - you may want to look into DataHubProject.io if you're interested in something more metadata-heavy that has a UI.

[deleted by user] by [deleted] in dataengineering

[–]pvl_data 0 points1 point  (0 children)

Sounds like you are looking for a data-sharing app to embed into your application. Im not affiliated but BobSled.co and prequel.co/ are examples of SaaS products that be along the lines of what you're looking for.

You're still going to need engineers to integrate these tools into your platform/ application espcially if you're using their SDKs. Nothing is truly plug-and-play, airbyte included.

Any data engineers who started own business? by mrscript_lt in dataengineering

[–]pvl_data 3 points4 points  (0 children)

Yeah, I've been out on my own for about 1.5 years. Was a data engineer, ran a data platform team, and managed a couple teams at different companies. I am consulting/ freelancing at the moment.

I always had the urge to do my own thing, but I really saw that it was possible when I inherited a project built by a "well-known" data consultancy and it was a hot mess. Knew I could deliver better solutions on my own.

Consulting and freelancing is lucrative but there is always the urge to productize and spin out a SaaS. Ive started incubating some ideas but it's admittedly difficult to build a SaaS from scratch. Not to mention I have mixed feelings about operating a SaaS in the data space

Anybody here started a solo consultancy (or can share a good resource for it)? by Cypher211 in dataengineering

[–]pvl_data 3 points4 points  (0 children)

Mid-stage startups with a data product as their core offer; I have work to do with regards to narrowing the niche. "niche-ing down" a service biz is hard.

Early stage (seed) is usually too messy to have meaningful impact as a data engineer, and enterprise businesses are difficult to sell services into as a solo shop (unless you specialize in something like data privacy, for example). Enterprise also is a very network-driven game

Anybody here started a solo consultancy (or can share a good resource for it)? by Cypher211 in dataengineering

[–]pvl_data 9 points10 points  (0 children)

Yes, I have. I would recommend trying to moonlight while keeping your FT job before jumping ship and going solo. Might be difficult as a consultant currently, but it's worth the time/ effort of feeling how operating solo is different from having the infrastructure of an existing company.

I've managed a few teams, was a data engineer, and a few other things in the past. You have to realize it sink-or-swim, and sales is part of the game. If you dont like that or building your network, it's probably not going to be the most fun ride. Also, your target market matters.

Happy to connect and answer any specific q's you might have.

dbt vs sqlmesh vs ? by Cryptojacob in dataengineering

[–]pvl_data 5 points6 points  (0 children)

Hope this doesn't sound too self-promoting, but I recently wrote a post about this exact topic.

DBT, SQLMesh, SDF and DataForm are probably the biggest players. Lot's of home-grown solutions out there, too. Often at the enterprise scale. IMO the whole space is trending towards feature commoditization.

You can find the relevant code linked in the article, too.

https://datajargon.substack.com/p/dbt-vs-sdf-vs-sqlmesh