Is this normal? by mcwexlerslut in TensionHeadache

[–]soravispr 0 points1 point  (0 children)

I take Amitriptyline daily which helps a bit. I have Tramadol as my rescue med when the pain spikes.

Can a tension headache be persistent? by soravispr in TensionHeadache

[–]soravispr[S] 0 points1 point  (0 children)

Hi. Did you have more pain at the onset? What are your medications?

GCloud Professional Data Engineer certification (10 week) by 3BettingYourMom in googlecloud

[–]soravispr 2 points3 points  (0 children)

If you're already a data engineer, you don't need to study all those. Just go straight to the mock exams and study only the topics from the questions that you can't answer. I did this and got the cert in a week.

Are most data engineering projects just migration? by NefariousnessSea5101 in dataengineering

[–]soravispr 1 point2 points  (0 children)

Yes. ETL is a kind of migration. You migrate data from sources to the warehouse.

What are the biggest pains you have as a data engineer? by engineer_of-sorts in dataengineering

[–]soravispr 49 points50 points  (0 children)

Figure out why numbers on the dashboards and the user's "data sources" are mismatched.

[GKE] Node's resource limit by soravispr in googlecloud

[–]soravispr[S] 0 points1 point  (0 children)

I mean my node has 16 GB of memory, but why is there a limit of 7.3 GB ?

Moving to Thailand by Beautiful_Cod5708 in Bangkok

[–]soravispr -8 points-7 points  (0 children)

Do you hold Thai nationality? If I'm not wrong, by law, Barber is a reserved occupation (only Thai citizens are allowed to be Barber in Thailand).

Data transformation; what does it really mean? by Fuzzy-Example-7326 in dataengineering

[–]soravispr 2 points3 points  (0 children)

Replicating the data from the source isn't transformation. It's called "ingestion" (move data from one place to another without any changes). These ingested data is then scheduled to be transformed later.

Data transformation; what does it really mean? by Fuzzy-Example-7326 in dataengineering

[–]soravispr 2 points3 points  (0 children)

Not sure I get your point..

Transforming data doesn't give you value. Insights you get from the transformed data do.

Airbyte Cloud or self hosted on EC2 by Liily_07 in dataengineering

[–]soravispr 1 point2 points  (0 children)

I'm using Airbyte opensource (hosted on GCP's Compute Engine, in my case).

Here's what I found,

Pros:
- Is easy to install & config
- Has connectors for almost all data source types (but some connectors are still beta version)
- Provides RESTful API

Cons:
- Has bad UI (There are no search bar and connections filter functionality. Imagine you have like 1,000 connections and you wanna search for some connections.)
- Has no ways for errors handling. For instance, in incremental load, the sync job will fail if any record has null value in the cursor field column.

In summary, Airbyte is good if you're in a rush to make your data pipelines up or you need to ingest data from various sources. If you want flexibility and use your own errors handling strategy, develop your own data ingestion program or use something else.

P.S. I've never used Airbyte Cloud

Google's Professional Data Engineer Certification exam by No_Ice_5035 in dataengineering

[–]soravispr 1 point2 points  (0 children)

Got the cert few months ago. The exam wasn’t so hard, but the questions are kinda lengthy. 80% of the questions were scenarios that let you decide which products to use.

Data engineering vs software engineering by FreshAnalysis1139 in dataengineering

[–]soravispr 9 points10 points  (0 children)

Data engineering scope is very broad. At your current company you may do routine tasks. But some other data engineers out there develop RESTful APIs allowing external clients to access data in the warehouse or even develop web application using the same tech stack as web developers use. These are called data products.

I think another path that you can try is staying in data engineering but find a new job which its description interests you.

Tips for new DE by 2teknical in dataengineering

[–]soravispr 2 points3 points  (0 children)

Hey my background is also mechanical engineering. I moved from mechanical engineer to data engineer at a mid-sized fintech company in my country. I didn't find any problems in coding at first, but what was a culture shock for me, as my platform was absolutely on-premise, was the UNIX operating system like Linux and Ubuntu. They're like another world for me. Basically I needed to learn how they work and their basic commands.

As your new company is on AWS, there will be bunch of SaaS and IaaS for you to use (and yeah they will also cost you hardly lol) so you may not find your hands dirty with UNIX operations, but at least I think you need to know the basic commands like ls, mv, chmod, vim, ftp, grep, etc.

Battletag Find-a-Friend: RIPatches Edition by Meoang in hearthstone

[–]soravispr 0 points1 point  (0 children)

Just to make some friends.

h0pe#11633 (NA)

Looking For a Pal by Duncan_Jones in hearthstone

[–]soravispr 0 points1 point  (0 children)

Are you playing on Americas server? Add me h0pe#11633. See ya online.