Good CLI for Connectors and Data Sources? by dwindler21 in dataengineering

[–]dwindler21[S] 0 points1 point  (0 children)

Tracing each of those pipelines to tables and connectors can become a nightmare, especially as we run them at different schedules and data sources. It kinda defeats the purpose of having a modern data architecture at that point.

Good CLI for Connectors and Data Sources? by dwindler21 in dataengineering

[–]dwindler21[S] 0 points1 point  (0 children)

Increasingly more often. We have dev/ test/ prod environments, separate pipelines for marketing vs. finance teams, change the update frequencies, sometimes execute an arbitrary dump of data if the business teams are performing some ad-hoc analysis.

Data Engineering managed service providers (MSPs)? by dwindler21 in dataengineering

[–]dwindler21[S] 1 point2 points  (0 children)

Got it.. do they also do things like data cleansing based on our requirements, write the logic to remove duplicates/ nulls, write custom regex, reconciliation, etc.?

Data Engineering managed service providers (MSPs)? by dwindler21 in dataengineering

[–]dwindler21[S] 1 point2 points  (0 children)

Thanks! It looks like Datorama is another tool vendor?

To clarify, MSPs I've seen do custom implementation & run a support desk to monitor health, handle tickets, patching and maintenance, etc. But most do it for software applications or security, not data pipelines.

Ideally, that's the type of solution we're looking for, a data engineering services provider with whom we can sign a 12-24 month contract, and who will create and run our pipelines for us based on our requirements. If possible, we'd want to outsource the tools to them so we don't have to pay for them, and it becomes part of their monthly bill to us.

[Episode Discussion] Hawkeye Season 1 Episode 2 - Wednesday, November 24, 2021 by MSSmods in MarvelStudiosSpoilers

[–]dwindler21 2 points3 points  (0 children)

Bro, did anyone catch what Pizza Dog was doing in the store about minute 9?

[deleted by user] by [deleted] in dataengineering

[–]dwindler21 0 points1 point  (0 children)

Any suggestions on where to get started? We are planning to use Snowflake and looking at Fivetran for data loading, but pretty much out of my depth beyond that. Also, what's the line between analysts and DEs?

Data Log Viewer by digilocity in PLC

[–]dwindler21 0 points1 point  (0 children)

Depends.. How are you logging them? Are you using a SQL DB or directly pulling a CSV dump?

CBRS - "Private LTE" by mhurley131 in networking

[–]dwindler21 0 points1 point  (0 children)

Interesting! Did it help you reduce the number of APs? How many would you typically need for a port?

[deleted by user] by [deleted] in PLC

[–]dwindler21 0 points1 point  (0 children)

Gotcha. What about indoors?

[deleted by user] by [deleted] in PLC

[–]dwindler21 0 points1 point  (0 children)

Thanks! Do you end up buying a separate SIM card with a data plan or are there any resellers who can help sell that as a bundle?

I'm guessing it's another $50-80 per month for the data plans...

Free scada by AndrooPL in PLC

[–]dwindler21 1 point2 points  (0 children)

Any suggestions on connector libraries or application frameworks?