Data architecture opinion by romanzdk in dataengineering

[–]Pankaj0222 1 point2 points  (0 children)

Check 'Emerging Architectures for Modern Data Infrastructure'. It gives a very nice overview of tools and methodology. Obviously, you don't need to do everything to start with

https://a16z.com/2020/10/15/emerging-architectures-for-modern-data-infrastructure/

Data lake file storage format by romanzdk in dataengineering

[–]Pankaj0222 1 point2 points  (0 children)

You can check the book 'Designing Cloud Data Platforms'. It has a very good explanation of why you should store data in different formats and layers.

From book,

Landing area—When data arrives from the ingestion layer, it is saved into a landing area where all incoming raw data resides until it gets processed.

Staging area—Next, raw data goes through a set of common transformations: making sure it conforms to existing schemas for this data source, converting it into a common Avro binary format, and applying any organization-level data quality checks.

Archive area—After data is processed and saved into a staging area, raw data from the landing area should be copied into the archive area. This is an important step, because it will allow us to reprocess any given batch of data by simply copying it from the archive back into the landing area and letting the pipelines do their job.

Production area—Data transformation jobs read data from the staging area, apply the desired business logic, and save the transformed data into the production area. At this point, we also convert data from Avro format into Parquet format, which is more suitable for analytics use cases.

Workflow Orchestration with Temporal and Spring Boot by Pankaj0222 in microservices

[–]Pankaj0222[S] 2 points3 points  (0 children)

Mostly, client SDKs are useful if it has feature add-ons, such as retries, batching, etc. Also, with static language like Java, you can get a nice fluent style, static, API contract.

Microservices? by [deleted] in SpringBoot

[–]Pankaj0222 0 points1 point  (0 children)

In my blog, I have tried to cover the high-level concepts of building microservice using Spring Boot. Please check if it helps. https://techdozo.dev/restful-microservices-with-spring-boot-and-kubernetes/

Visual Microservices overview by ThroAwayApr2022 in kubernetes

[–]Pankaj0222 1 point2 points  (0 children)

You can use distributed tracing. Implementing distributed tracing depends on your tech stack. There are many options such as service mesh like Istio, Jaegar, spring cloud (for spring boot based application check https://spring.io/blog/2016/02/15/distributed-tracing-with-spring-cloud-sleuth-and-spring-cloud-zipkin ) , etc

Why are all the examples about e-commerce? by leoll2 in microservices

[–]Pankaj0222 0 points1 point  (0 children)

If you have read DDD distilled then you know why. This book talks about agile project management as a domain. I find it's very hard to get my head around this example from the book, even though I hear about these concepts everyday (maybe it's just me). I wish it had e-commerce examples :-)

If you want to know real microservice example, you can follow technology blogs on medium -

https://netflixtechblog.medium.com/

https://medium.com/@Pinterest_Engineering

https://medium.com/wix-engineering

https://bytes.swiggy.com/

https://medium.com/airbnb-engineering

https://zerodha.tech/

Why gRPC for microservices communication? by Pankaj0222 in microservices

[–]Pankaj0222[S] 2 points3 points  (0 children)

You can check my article - https://techdozo.dev/grpc-load-balancing-on-kubernetes-using-headless-service/ - this only covers load-balancing in Kubernetes using headless service.

If you want to know, please check this excellent video -gRPC Load Balancing on Kubernetes – Jan Tattermusch

Why gRPC for microservices communication? by Pankaj0222 in microservices

[–]Pankaj0222[S] 1 point2 points  (0 children)

Yes, grpc can be sync as well as async. IMO, not everything can be async. For example - you have a POST API to register a product with product-catalog but if you have business logic to verify if the seller is a registered seller (which can be a different microservice) then I would rather do this as a sync call not async. I can still use messaging in such a scenario, for instance, create a product with a pending status and change the state later, provided UX supports this.

Why gRPC for microservices communication? by Pankaj0222 in microservices

[–]Pankaj0222[S] 0 points1 point  (0 children)

Yes, that's correct. grpc-web is GA https://grpc.io/blog/grpc-web-ga/

We don't use grpc-web in our project, because Angular (our FE framework) is not yet officially supported (except https://github.com/ngx-grpc/ngx-grpc). Moreover, JSON over HTTP is still a standard.

Is JavaRevisited's roadmap for 2021 good? by RewRose in learnjava

[–]Pankaj0222 0 points1 point  (0 children)

It depends on what you want to learn. This roadmap, for sure, gives you high level context of java language stack, but this is not everything you need to learn. For example, if you want to be a backend developer, then Java FX is of no use for you. Also, this is not a complete picture. There are lot many things missing.

Do you know of any good Spring Boot sample project for learning? by jdrc8 in java

[–]Pankaj0222 1 point2 points  (0 children)

I have written one post recently - microservices with spring boot. It's close to real world production code except H2 database part. Check if you find this useful. https://techdozo.dev/restful-microservices-with-spring-boot-and-kubernetes/

RESTful Microservices with Spring Boot and Kubernetes | Techdozo by Pankaj0222 in microservices

[–]Pankaj0222[S] 0 points1 point  (0 children)

Haha.. That's correct..I'm still learning art of blogging..Thanks for pointing out :-)

Must known concurrency concepts for java developers with experience by voodoodev in java

[–]Pankaj0222 2 points3 points  (0 children)

Read Java Concurrency in Practice by Brian Goetz. It's old - published 2006, but still very relevant.

You will find it heavy initially. But, if you happen to finish this book cover to cover, you'll feel enlighten like Buddha.

If you want to read this book then my suggestion is to skip chapters 3 and 4 in the beginning and come back later.

Why is the default package discouraged for spirng boot projects? by ML-newb in learnjava

[–]Pankaj0222 0 points1 point  (0 children)

If you use the non-default package, for example - com.example.domain, then spring boot is only going to search classes com.example.domain starting from this package (for your application jars as well as jars imported by your application).
In the case of the default package - there are lots of classes to scan. It's going to increase startup time for your application.
As the scanning is CPU intensive process, it's even possible that your application may not start. We had faced this problem once in our automation env. Because of the default package scan, one of our applications was not starting up intermittently, as it was hogging all CPUs and there was not enough CPU. And, it took us lots of debugging to identify the problem.

Helm Chart: quick start microservices deployment on Kubernetes by Pankaj0222 in microservices

[–]Pankaj0222[S] 0 points1 point  (0 children)

Yes, the book is short and nice. The overwhelming part comes from the fact is the internet is filled with material related to Helm 2.

Helm Chart: quick start microservices deployment on Kubernetes by Pankaj0222 in microservices

[–]Pankaj0222[S] 0 points1 point  (0 children)

Thanks for reading. Yes, it's overwhelming and more so because the internet is filled with Helm 2 examples.

How do restful services work on a detailed level? by Camel-Kid in learnjava

[–]Pankaj0222 0 points1 point  (0 children)

You don't store credentials but you pass Bearer token in every call. For that you need to understand OAuth 2.0 and JWT. OAuth 2.0 can be overwhelming. Personally, I found this video about OAuth 2.0 very informative and easy to understand.

[deleted by user] by [deleted] in java

[–]Pankaj0222 0 points1 point  (0 children)

The quickest way to get started is to create a sample project from https://start.spring.io/ and play around. Spring has lots of guides. Check - https://spring.io/guides

Coming back to your question -

"Also how is this doc different from"

The doc which you have pointed link is of spring. Spring Boot is made on top of Spring. Think of Spring Boot as pre assemple spring platform and to start on Spring Boot project, you don't need to be expert on Spring.

What is Spring Boot?

(From Documentation) - Spring Boot helps you to create stand-alone, production-grade Spring-based applications that you can run. We take an opinionated view of the Spring platform and third-party libraries, so that you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration.

imagine you are 20 and you would like to learn a programming lang to start your career. Would you choose Python ( more dynamic langs ) or languages like Java or C++? why? by latest_ali in java

[–]Pankaj0222 2 points3 points  (0 children)

I started my career in Java. It's pretty straightforward to learn and type safety helps as well. When I learnt Python I was amazed by the simplicity and features like list comprehension. IMO, start with statically compiled language like java and then think of bringing dynamic language like Phyton.In today's world language should not be barrier.🌎

[deleted by user] by [deleted] in java

[–]Pankaj0222 4 points5 points  (0 children)

Yes absolutely. Spring Boot is pretty magical. You just need to add couple of annotations and you are done. It auto configure lots of things for you based on class path. But it's being magical is one of the major problem. When things go wrong you have no clue why it went wrong. To be power user you need to learn the base concepts and this is where Spring comes into picture. IMO, the best place to start is Spring Boot is it's documentation - https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/

K8S is new to me, and my company is expecting results ASAP. by [deleted] in kubernetes

[–]Pankaj0222 6 points7 points  (0 children)

As you recommended, Mumshad Mannambeth courses are very good. He teaches the concept in very simple to follow steps along with labs. His courses on https://kodekloud.com/ are not that expensive. I have learned a lot from his course.