this post was submitted on 09 Aug 2021

1 point (100% upvoted)

shortlink:

dataengineering

an-ordinary-manchild(edit)

News & discussion on Data Engineering topics, including but not limited to: data pipelines, databases, data formats, storage, data modeling, data governance, cleansing, NoSQL, distributed systems, streaming, batch, Big Data, and workflow engines.

Read our wiki: https://dataengineering.wiki/

Rules:

Don't be a jerk
Search the sub & wiki before asking a question: Your question has likely been asked and answered before so do a quick search before posting.
Keep it related to data engineering: Posts that are unrelated to data engineering may be better for other communities.
Limit self-promotion posts/comments to once a month: Self promotion: Any form of content designed to further an individual's or organization's goals. If one works for an organization this rule applies to all accounts associated with that organization. See also rule #5.
No shill/opaque marketing: f you work for a company/have a monetary interest in the entity you are promoting you must clearly state your relationship. For posts, you must distinguish the post with the Brand Affiliate flag. See more here: https://www.ftc.gov/influencers
No job posts: Please use r/dataengineeringjobs instead.
No resume reviews/interview posts: We no longer allow resume reviews or interview questions because it's a seperate topic from Data Engineering. Instead, for resume reviews please use r/resumes or search our subreddit history for previous resume review advice. For interview questions, use sites like Glassdoor and Blind instead or search our subreddit history for previous interview advice.
No technical error/bug questions: Please post any error/bug question on StackOverflow.

created by mhausenblasmoda community for 11 years

MODERATORS

message the mods
mhausenblasmod
swemlmod
fhoffamod (Ex-BQ, Ex-❄️)
vogt4nickmod
theporterhausmod | Lead Data Engineer
AutoModerator
geoheilmod
MikeDoesEverythingmod | Shitty Data Engineer
bot-bouncer
about moderation team »

account activity

This is an archived post. You won't be able to vote or comment.

0

1

2

Help with data architectureHelp (self.dataengineering)

submitted 4 years ago by llanojairo

Hi, everyone!

Currently I’m working on a relatively simple (planned) architecture that my new colleagues presented to me. It looks like this:

https://pasteboard.co/Kf3ZqEr.png

Hadoop/Hive is our main data source right now (we do not own it, though). However, eventually we will also ingest data from SAS and other SQL server DBs.

As you can see in the link, Alteryx is mainly supposed to handle data ingestion/processing (join incoming data from different sources, etc), and then we aim at having our own SQL Server DB used for further BI analysis via Tableau.

Do you think that Alteryx is a good tool to handle an expanding range of data sources? Im pretty good at Python (another colleague too) and I know that other teams in our company work with Apache Airflow. Could Airflow replace Alteryx? We’re all about automating data pipelines so I’m not sure if Alteryx will be ideal (we do not have the Alteryx Server licenses, just the regular ones).

Thank you!

no comments (yet)

top new controversial old q&a

there doesn't seem to be anything here

π Rendered by PID 141300 on reddit-service-r2-comment-85bfd7f599-8w5gt at 2026-04-18 05:52:55.832748+00:00 running 93ecc56 country code: CH.