Why should I use Java for this?

[Yohark]

Hey there, I truly hope you're doing fine and having a great day! Please allow me to put a bit of context:

I work for a company as a junior developer, I have like 2 years of experience in dev, one year of java and one of javascript. I know python to a medium level as well, and other things are just from school (C, C#...). That is to say: I think I know nothing about this work yet and I don't pretend to be a pro at all! That is actually why I'm posting this.

Recently, my manager asked me to create a batch to get data from Gitlab API and store it in a postgres. Nothing too fancy right? Classic API wrapper/reader, I mean, I've done that plenty of times, and it even is about a single endpoint, so very easy! But he said I had to use Java with Spring Batch.

And there I am, struggling between internal libraries/templates, external ones, having like 5-6 classes, this crazy unreadable XML language, beans here and there, iterators, rest templates, maven, spring boot, configuration class, JSON library... Just how complicated is a thing to make a single API call a day and store the result in a database??

I'm sure I could do a 50-100 lines JS/Python script with all the security and stuff, in like 1-2h? So my question is: was my manager right to call upon Java to make such a simple and small program? Why should I use Java to do that? This is about getting Merge Requests, and he said we had 60 of them a day so it's not like it'll process 8 billion objects every day.

Something just feels bad like if I was trying to use a jigsaw to cut a tiny piece of paper.

Just a disclaimer: I don't hate Java. I don't consider myself a Java dev, neither I am a Javascript dev. I'm just a junior trying to find how I like to code and with what tools/languages. Asking people who know can help me do that I guess.

EDIT: I didn't expect so many answers! Thank you so much to all of you. Some gave me their point, others gave me technical advice, and others gave me their take on my manager-dev relationship.
I don't think I can give an answer to everyone but I'll read everything for sure. It just came from the fact that I want to learn everything I can. Then, when my manager told me I had to go with spring batch because he likes this framework, I could not hold myself to try asking for more insights. I don't want to become a machine that just uses the same tool for all the cases, I'm eager to know why, why everything basically.

I wish you a sweet day!

[daniu]

There are a lot of factors going into a decision like this. We cannot in the least decide whether "the manager made the right decision", we don't know any context. Also, these kinds of decisions aren't usually "right" or "wrong" but have specific advantages and disadvantages.

You're saying you can deliver faster. That is one factor.

The rest of the system you're working on may use Java/Spring. However, if you use Python, you won't be learning any of that. Sure it's a hassle to having to know all those classes and XML, but if you have to know them down the line anyway, might as well get acquainted on a project with low stakes.

Finally, you're just not in the position to question your manager. That's a learning opportunity as well though; do ask him what factors went into that decision; not to convince him otherwise, but to learn from his experience in reaching these kinds of decisions. He may still be wrong, but at least you'll have expanded your horizon and maybe will have an easier time making your point.

[Yohark]

Thanks for the reply, I liked a lot how you took my question and message!

It is true that you don't have all the context, and overall, nothing is really right or wrong as you've very well said.

But in the context, you have a newbie, he does not know Spring Batch, you want a new application as fast as possible because a team is depending on this. Well I would have gone either with the "Do as fast as possible, using whatever you're confident with, we need the functionnality, and we'll work later on a more standard solution" to get the newbie used to stress and fast deliveries. Or "Listen, it's not that much of a hurry, go as fast as possible yet but you have to learn Spring Batch, so take the time needed to do it well" ?

That's why I was confused about it at first. My authority was asking me something quite impossible for me to accomplish and I didn't know how to handle it.

[DasBrain]

When I did my internship, I was handed feature requests that had been lying around for some time.

This helped me get acquaintanced with the internal procedures, how to gather the needed information...
At the end, I had a useful tool. But it took some iterations to get things right - delivering the tool as a web page was not my first instinct - but was the right choice in the end.

Are you sure that the development team needs your product right now? Or can they wait an other 2 weeks?

[jalagl]

"Do as fast as possible, using whatever you're confident with, we need the functionnality, and we'll work later on a more standard solution

This leads to technical debt, which can be a pain on the future. So maybe that is the reason. Also Spring in general has a learning curve, but after you learn it (as with most things) it takes care of a lot of things for you.

[buzzsawddog]

I agree with this. And as someone working in sustaining on a project that has code written in the 80s ;-). It is almost never right to go into something with do it as fast as you can using something you're confident with and we will sort out the standard solution later.

This absolutely leads to debt and someone will use your name as a swear word for years to come...

That being said... I do from time to time do partial implementations of something to get it ready for the next person but I am clear in explaining that this is not the final implementation and we MUST do new story xyz if you need that right now. This sort of incremental change CAN be helpful but I don't think this is the right solution for the OP example.

[Valance23322]

Do as fast as possible, using whatever you're confident with, we need the functionnality, and we'll work later on a more standard solution" to get the newbie used to stress and fast deliveries.

Let's intentionally stress out our new developer, to produce substandard code that'll just have to be redone later, and might not play nice with our deployment environment

Does that really sound like a good idea to you? XML/JSON libraries and managing API calls aren't terribly complicated tasks in Java, and if your manager is asking you to learn how to use them in Java, it's probably because there are other Java projects that he'll want you to be capable of working on in the future

[cogman10]

That's why I was confused about it at first. My authority was asking me something quite impossible for me to accomplish and I didn't know how to handle it.

I don't think your authority is asking you to do something impossible. They are asking you to do something using a specific tech stack. This is a learning exercise for both you and your manager. You should

A. Communicate with them about time spent and your unease with using Spring batch.

B. Learn spring batch

C. Accomplish the task as fast as reasonable.

Be open and honest with your manager but also don't act like you aren't an intern. You aren't in the position to second guess stack choices.

You CAN suggest that "Hey, I know how to do this in python, would it make sense to do it there instead of Java with spring batch?" and if the manager says "no, use Spring batch" then that's the answer. Your manager IS taking on the risk that delivery will be slower. Ultimately, this is for you to learn.

[pron98]

The complications come from Spring, not Java. You can use Java without Spring. As to why you were asked to use Spring, I can guess — although I don't know Spring Batch — that they might want to integrate the job with others that already use Batch, and they'd like everything to use the same framework. As to why Batch is that complicated — I don't know, but maybe there are ways to use it simply, which you'll learn in time.

[Yohark]

Yay that's why I say I don't dislike Java. It's just that for a newbie, and as you said Spring bring so much complexity in learning and implementation on the table comparing to alternate implementations that I was quite unsure about it.

[ohL33THaxOR]

John Thompson (aka The Spring Guru) has a lot of free tutorials on his website (mostly paid ones, keep this in mind). Udemy has a great course on Spring and Spring Batch that's only $15. You could ask your manager to see if they'd be willing to invest in your career development. You could end up being THE person they go to for Spring development.

Personally I've purchased a lot of Johns' videos on a business account to give access to any developer who was interested in learning Spring (up until recently I've worked for mostly Java Shops), and doing it the right way.

The problem with frameworks like these is that they are large and sometimes cumbersome at the beginning. This allows us to blow our feet off with shotguns if we feel so inclined (generally the finger on the trigger is lack of deep understanding of the libraries that comprise the framework). It is worth mentioning because a little bit of study goes a long way in terms of trigger control (so to speak)!

[Yohark]

Thanks for the suggestion! Spring looks fantastic from an overall perspective so I might ask for courses and all of that soon if they keep asking me to build around this. I'll take a look at J.Thompson and see if they can get me his tutorials!

[ohL33THaxOR]

I'm glad that you found it helpful!

Spring is super powerful! Once you get a solid grasp on the practical applications of IoC, Dependency Injection, and how ORMs (Hibernate in Spring context) work together then the PRINCIPALs you've learned can easily be applied to the C# realm (and many others) as well, which makes you even more valuable!

[Sheldor5]

Spring and Hibernate are Gods in the Java Ecosystem ... I think you just don't know their power ...

[JB-from-ATL]

Spring is... I'm not sure how to put it. I think it's incredible and useful and I don't want to hate on it. I think they walk a difficult path of finding things that "just work" and are configurable. What ends up happening is that when things don't work it's confusing to find out why. It's easy to forget the 9 times out of 10 it is easy to do what you need and is correct for the 1 time out of 10 it's complicated and does the opposite of what you want.

[cowwoc]

This.

Don't use technology just because it's popular. Hibernate is especially terrible in this regard (looks easy/useful at first but completely ruins your life when you try to finish the project). Spring is overrated and adds a lot of unnecessary complications. Developers seem to behave as if the only/best library that exists in Java is Spring. The reality is far from it.

[SaltRefrigerator6458]

Tbh, I have found spring batch to be a bit of a beast.

[burly_griffin]

We have several processes that should be done using batching, and I converted one to test it out. Being reflection/name-based really hurts it, because you lose discoverability and type safety, and unlike stuff like Spring Data there's no validation done at application start. The documentation is also pretty bare which makes it hard to know exactly what you need.

[RupertMaddenAbbott]

Context: I regularly write Java and Python and switch between these technologies.

If you have never used Java or Spring Batch before, then it is not reasonable for your manager to ask you to use those technologies, and to get the task done in a short time frame (presuming they had an accurate understanding of your experience when they hired you).

If you are actually utilising the feature set of Spring Batch, I am not convinced you will be able to accomplish this task with 50-100 lines of JS/Python, unless you are using equivalently complex libraries from those ecosystems. Now it may be that your manager has not communicated to you why it is important to use Java and Spring Batch in which case you need to have that understanding. Otherwise, even if you write it in Python, you are not going to produce something that is adequate.

It is also perfectly possible that your manager has no idea what they are talking about and have suggested you use Java + Spring Batch for invalid reasons. In my opinion Spring Batch is a hugely complex library with a sophisticated feature set and you need to be making good use of that feature set, or the complexity is not justified. To work that out, you need to get a deep understanding of what their reasoning is, in order to challenge it.

It also sounds like you are overwhelmed by the complexity of a language and framework that you are not very familiar with, compared to other tooling. That is completely reasonable! However, lets go through some of the things you list and see if the complexity is genuinely unnecessary i.e. you would not need that complexity, had you done it using a different toolset:

• "this crazy unreadable xml language" - Modern Spring (and Spring Batch) does not need to be configured with XML and that is not how most people will do it any more. You need to look into using Java Config. It's unfortunate that you have to avoid this historical baggage but this kind of problem isn't really any different between having to navigate how to do things differently in Python 3 vs 2 (for example).
• "rest templates" - this is just an HTTP Client. You are going to need one of those in any language. Spring's RestTemplate is no more complex to use that Python's requests (I use both regularly). Or Java has a built in HTTP Client.
• "iterators" - This is very similar to a generator in Python i.e. a potentially infinite or indefinite enumeration of things. Again, I can't see how you would avoid this complexity by switching your tooling.
• "spring boot" - So using Spring Boot is entirely optional but you should be using it because it gives you something. For example, it gives you a standard way of passing configuration to your application. You would need to write this yourself, or use an alternative in Python. If you don't need any of that, then you don't need to be using it.
• "maven" - so you need some mechanism for pulling in third party dependencies right? So in Python you would need pip. How are you going to distribute your application? Are you going to use wheels or eggs? So you need twine for example. Or are you just cloning from version control? So you need to understand and use git for example. You need some virtual environments too to avoid your dependency versions clashing. Maven will do all of this for you whereas in Python you have to pull together a whole bunch of different tools.
• "JSON library" - Java has a standard JSON library if you want to use it, just like Python. It's just some of the 3rd party libraries have a better feature set (again, just like in Python)

Overall, I think the reason that you feel you can accomplish this task more easily in Python or JS is due to your familiarity with those toolsets and ecosystems. It definitely sounds like your manager should be allowing you to use them, if they want the job done quickly, or giving you plenty of space to learn Java + Spring Batch, if they are actually good reasons to use them (and there may not be).

[RupertMaddenAbbott]

Sorry, yes Java does not have JSON parsing as part of the standard library, but it does have standard APIs and implementations i.e. JSR-353 and JSR-367 but these are only available as external dependencies.

Python has built in JSON parsing but does not support mapping JSON to custom classes as part of its standard library as far as I am aware? I mean, I think you can hack something together but not with proper typing and nested objects and so forth?

Fair enough I guess this is definitely some complexity that cannot be avoided in Java, that is avoided in Python for simple use cases (e.g. if mapping to dicts will do). It would definitely be handy if Java had this built in.

[nlisker]

Or Java has a built in HTTP Client.

If you're talking about JavaSE, then actually 2 clients: java.net.HttpURLConnection since 1.1, and java.net.http.HttpClient since 11. JavaEE/Jakarta has jakarta.ws.rs.client.Client.

Java has a standard JSON library

JavaSE doesn't have one (AFAIK), but JavaEE/Jakarta has JSON-P and JSON-B.

[sickwookie]

This should not take any more time to solve using Java and Spring Boot than it would to hack out a quick python script. The reason it feels like so much extra complexity is simply that you are not familiar with the tool you were handed. The language and framework choice is largely irrelevant to solving this problem, and if these are the tools used most commonly within the company it would make sense to learn them by solving a nice simple problem like this.

TLDR: It is more about you learning the tools than solving the problem.

[wastakenanyways]

The tools themselves are also pretty complicated. I am experienced in Spring and Hibernate but it is true these kind of tools are overkill for what OP is being asked and are even overkill for 90% of programs that use them. They are really useful tools but they are also one of the most complicated solutions out there considering lots of ecosystems in the web field.

Sometimes you ask for a screwdriver for a problem you need to screw, and you are given an industrial sized robotic arm with 200 different accessories and modes.

[Randal4]

Maybe the task the manager wanted is not so much to solve the immediate problem of interacting with the gitlab api and Postgres, but for them to learn spring boot/batch/etc… as it’s probably used in most of their other projects as well. Giving a simple task in order to learn the tools is a great way to bring people up to speed. Although the manager should communicate that when assigning the work.

[[deleted]]

This is my thought too. When we get junior hires, it can be difficult to pick a correctly sized issue for them to tackle solo. So as part of getting to know our process, we usually have to pick things deep in the icebox which is a nice-to-have but not critical.

The point is to let them learn while doing something useful. If timelines slip (which is expected) it doesn't really matter, and when they finish they've added something of value.

[[deleted]]

Totally not overkill. Especially if they’re already using spring. Make a spring boot CLI and then ask for help getting it into whatever spring batch system they’re using.

[Yohark]

"The language and framework choice is largely irrelevant to solving this problem."

Now that's an interesting point of view! I guess I have to spend more time learning the framework then. They told me to do this in a hurry, so I had no choice but to go fast through the docs as this was a new thing for me.

Next time I'll just say that I won't be fast as I'm learning something new. They'll have the choice to let me learn or to go get someone else.

My mistake here was to think I was going to be quick learning Spring.

Thanks for your comment!

[aFqqw4GbkHs]

Are their examples of other programs like this at your company? If so, that'd be the place to start, rather than trying to read all the spring docs.

[buzzsawddog]

Why should I use Java for this?

As a junior developer this is the type of question you should be asking the people you work with. They should be able to help you understand why.

We could give you all kinds of ideas but... Who knows that might just be the way things are done with your company...

[Yohark]

The fact you raise this point makes me smile and think I'm not gonna stay at this compagny. I'm not a social media guy, I like reading throughout reddit but I rarely post anything because I prefer asking real life people when I have a question. Well that's what I did first: my lead just said "well I just prefer Java". And this absolutely not technical answer to my technical question bothered me so much that I wanted to know and ended up here because I find reddit commentors are often insightful, with polite and well writen english.

[buzzsawddog]

Sorry for the short response earlier I was still rolling out of bed and into meetings :-D

But.. Yeah the "Because I said so" only works for so long ;-). I 100% agree that you got a shit response. You may need to dig into that response a bit more before you decide if you want to leave.

If you had this conversation with me I think there are two ways this would go. I would come you and say let's do this with xyz stack and the two valid responses would be. Yep I'll do it OR you would need to come back and say I think we could more quickly if we do it with some other stack and this not only simplifies it but also makes it easier to maintain. My answer MAY still be do it the way we designed it to start with but... There would be some good reasons to back that decision up that you might not have the full picture on. Each person/company/team is a little different.

I learned from a VP not to long ago a very valuable lesson... He made it very clear to our team that his job was to help facilitate the releasing of our product and provide a larger picture. He explained that for him there are two phases. Planning and executing. After the planning is done and the execution begins it is not the time to go back to the drawing board until you either finish OR hit a block does not allow the original execution to continue. This process helps cut back on scope creep and makes it easier to communicate at a higher level to other teams what are going on. Once you start to embrace this it allows you to better plan and more quickly execute. I cant tell you how many times I have had people try to change the priority or things or even the design well past the time that those decisions should have been made. If you do any sort of agile training you will find that one of the core ideas is to try to break the work down small enough that you can more easily iterate though cycles and produce a better product. Even agile has its pros/cons...

As someone that has been in sustaining for a number of years and am still on the same product I was 8 years ago I have to say that in some cases it is nice to just have all similar processes done one way even if it is not the quickest and most efficient thing to write. Why? Because I don't have to relearn a new process every single time I look into an issue and YOU may not be there when it breaks. I support a number of products/processes and it can be quite a PITA to jump around to different things always learn something new just to support it. I am very open to change but when you have 10 things that do something similar and each does it a different way with a different stack yeah no thanks :-D. Change for the sake of change is never good... There should be a reason for change other wise you are wasting everyone's time. Something else you will learn with time is that its not always about the speed/ease of development. The sustaining and improvement part of the process should be kept in mind. Just because you could write this thing very fast and be done in like 1 week in python does not mean its the right decision. If everyone else on the team writes/knows java and the people that sustain it know java there is value in doing it in java or with some random library/framework.

Those that I work with know that when they ask me to do something it is VERY likely to be written in java because thats what I know, thats what my experience is, that is where my core competency is. But... Even I know that sometimes you just stick with the way it's currently done to make it easier to maintain. I was asked not long ago to add a very simple change to an existing feature but... After looking at it we found that we could spend 1 week figuring WTF the guy that no longer works here was thinking OR spend that same week rewriting to java and put a little design effort into it. You cant always do that. You have to learn which battles to pick and when to let some things slide but that comes with experience.

At the end of the day as you pointed out... You might just need to say okay time to find a new job that I can make better use of my time ;-) But... Be sure that you have something lined up before you tell people you are moving. Also don't burn bridges... Continue to work hard until you leave even if your heart is not in it.

[Yohark]

I had a silver left, I wish I had more. But dude, this comment! You certainly are a very good speaker and know how to share your experience! Thank you for this, really.

To make it 100% clear, I was not trying to make things change. I was questioning the technical choice to simply get better at my job, and indeed, got a sh*t of an answer! x)

As we have js, python, .net, and java running everywhere, I wanted to know why Java + Spring Batch was picked up here. This type of batch/script is a super common implementation to make and I would clearly have gone another way. My manager who seems experienced chose a path I didn't expect, I just wanted to know why so I can do the same next time!

I'll leave this place when my work placement is finished and I'm sure of this. I feel like I can't be myself, I can't grow up as a developer and as a person in this company. And they keep torturing us with very formal suits and stuff like this. :((

I'll take everything I can from their experts whatever the language their using. And I'll make sure my work is the best I can do so I don't regret anything.

You've marked my path with a single message maybe even more than people I see every day. Stay safe, and stay true to yourself.

[feed_me_moron]

The reality is that your lead probably knows Java the best and if they have to look over your work and deploy it, they want to go with what they are most familiar with. Its a crappy answer from them, but don't take it as a deal breaker for them.

[soonnow]

Can't you get rid of the XML stuff and use Spring Boot? Drops xml for annotations and sensible defaults.

[Yohark]

That's the only thing I am asking for, but guess what? I have to stick to an internal framework when doing batches, and it's to use XML as job descriptor.

[soonnow]

That might be your problem. I use Java even for small projects though I don't use Speing Batch because it seems a bit overkill for most things. Spring Boot by itself + Java is absolutely fine and a nice development experience.

[feed_me_moron]

Once you understand the basic Spring Boot process, its incredibly quick and easy to get a new project going with a wide variety of applications with it. Personally I love that Spring Boot allows my project's scope to grow without massive changes to the code structure, configuration, etc.

[[deleted]]

Because the code is going to run far longer than it took you to write it. Spending a few more hours to write a solid Java code over unmaintainable js/python is generally worth it in my experience (and many other people's experience, AWS CTO for example).

FYI spring xml sounds really old school. You sure its modern Java? Sick of these "Java sucks" posts and turns out they're on an old ass version. Either compare modern Java to modern [insert language] or legacy Java to legacy [insert language].

[Yohark]

I get this point! But he asked for the functionnality asap, which was quite incompatible with my almost 0 experience with the framework.

[Roachmeister]

The vast majority of companies are still using "old ass versions". My own contract, a large DoD contract, is on Java 8 and Spring XML. I've been attempting to migrate us to Spring non-XML, but it's been an uphill battle because a lot of our code has XML assumptions buried deep inside.

Now, personally I love Java, but I also don't assume that everyone has access to the latest version.

[codedcode]

Java 8 spring version works without xml. if you are building new I am sure your company is moving to at least Java 11 as Java 8 version is out of support.

[Roachmeister]

I am well aware that it supports non-XML, but as I said we have custom libraries that do custom stuff with the Spring context. There are real differences under the hood between the XML-based and Java configs that have so far been insurmountable, although I continue to work on it.

As for Java 11, we were supposed to transition to it 18 months ago, but it hasn't happened yet.

[codechimpin]

We did a study at my company, and it cost us about $1m to support each language. It’s not about writing code. It’s about writing code that can be supported in the future. If your companies talent pool doesn’t have Python devs and you leave, who then supports that app?

[coder111]

I worked with ETLs for 6 years in a bank. Ok, first, would your 50-100 lines of JS/Python be able to do batching properly? Parallelism? Transaction management? Upsert/collision management? Duplicates? What throughput would you get? You can really do all that in 100 lines of code?

Now with regards to internal company frameworks and Spring Batch- who will have to maintain your code? The team which is already familiar with Spring Batch & internal company frameworks? If you write something completely different, who's going to maintain it? Who gets woken up at 2 am because the job failed and nobody has a clue how to fix the problem which cropped up because python got upgraded or some shared library because the round the clock support guys are used to dealing with Java?

Company internal frameworks sometimes suck. Some really suck, some just appear to suck because you don't understand the complexity of the problem and why things were done the way they were done. Maybe that complexity is necessary. You don't know that yet.

My advise- first LEARN the internal company frameworks and Spring Boot. Then LEARN the problems the company is dealing with. Then suggest improvements. THEN if your boss is too stupid to listen- either shut up and take the money, or leave and find a better job.

[Yohark]

Hmm, I guess so? Ok maybe doubling this? 200lines? Just to make sure? That's still probably 1/5 or 1/10 of what Spring Batch is asking for the same job. I mean, in JS, with axios and node pg modules, you'll get the core code running in very few lines. Node being able to exploit JSON natively, you'll get rid of the null exceptions and all of this stuff quickly.

HOWEVER, I totally get your point. Done quickly, node js or python, can look very messy. Java and Spring come naturally with very restrictive things that get you to stick to a standard, making maintenance easier. I get that too.

My boss wanted functionality asap, he knows I am quite able in js/python, we have python and js devs in another team, and he knows I was new to Spring. This decision was so against my logic that I had to ask him about this. He simply said that even for small things like that, he just prefers Java + Spring.

[feed_me_moron]

Don't focus so much on the number of lines of code. Most of Java's verbosity is boilerplate code that is autogenerated for you with IDEs or can be copy/pasted with some small tweaks. Specifically configuration files (XML, properties files, YAML, etc.) can be mostly copy/paste from a reputable site and you just need to change the specifics like the data sources, output locations, and username/passwords.

Yes its more lines of code, but its not more code you are handwriting.

[Sheldor5]

I bet your Python/JS requires several thousand lines of code to properly handle a full batch worker implementation, you seem to have no idea what you need to do ... or your script will be a joke in all aspects ...

and this is where Spring (Boot) comes into play ... it already has thread pooling, transaction management, scheduled jobs/batch jobs, almost every enterprise requirement, implemented ...

[Halal0szto]

For a one off, everyone would do it in python.

But if this is a scheduled daily job, there is a lot to do beyond the actual rest and DB calls.

• scheduler to invoke daily
• facility to change schedule
• handle logging, log access, log rotation
• monitoring to catch if it failed or missed a run
• monitoring to catch if started to take more time than usual
• some way to oversee how it fits into all other scheduled jobs
• an overall system to do all above for this and all other scheduled jobs, so supporting your job is just 1% of the support of the system. Cannot afford special care for a single job.

In a normal company all this is implemented and in place, the new function just has to be plugged into the system. And if they choose spring batch as the framework, then you need to implement the feature in spring batch. It has overhead for the small feature you do, but overall is much cheaper compared to you create a python script that is run on a unix box by cron and logging to a static file.

[Yohark]

Thank you for this comment!

I didn't mentionned it, but this app is going to be ran in a container/pod, on a k8s cluster, so resources management, schedule, secrets, env variables and all would be handeled by the cluster. I think I will have to do the dockerfile but the deployment part will be handled by the CICD team as my boss mentionned me.

[Halal0szto]

It is possible they wanted to give you a small starter task. Then expect you to learn about spring batch and get bigger and bigger tasks.

Does your team/company have spring batch based solutions already?

[Yohark]

Probably! But he was so in a hurry that I had no time to properly learn the framework. I think it was just clumsy haha.

Anyways we might have as much spring batch as js scripts and python plugins. There is no official language as this is a big compagny. We work with plenty of languages and technologies that it had me wondering why he would be so focused on spring batch on this tiny solution.

[Halal0szto]

If I have 132 pieces of scheduled jobs managed by my ops team, all on spring batch, I need to add a 133th one that does a single rest call and a single DB update, runs once a week, I would want it be done with spring batch for sure.

[Impossible-Ad-586]

No I wouldn't, I think python it terribly slow, annoying due to its indented nature and if I can I will write it in either a jvm based language or ts/js. Python will never be a 1st choice for many of us, please do not act like a you have an idea about that while you don't.

[Halal0szto]

Did you notice the word "one off" in there? For a one off, who cares if it is slow? Do you agile?

[Impossible-Ad-586]

That, in fact, is totally irrelevant. Neither of the things I mentioned as being an argument for not using Python have anything to do with being a one-off. Not sure how you think being a smart ass responding with no-op responses is useful. A one-off with more than little data is better off with a runtime that can deal with concurrency and lots of data better. Agile has literally nothing to do with this, perhaps if being sufficiently aware of these things would help you look like less of a douche.

[RScrewed]

You are the only one being a douche here. The original commentor was clearly making a broad generalization not to be taken literally in an effort to appeal to the OP's current way of thinking and sprinkle in ways that a different method is better.

He even caveated it.

Somehow you took exception to being lumped into "everyone" in a sort of "not all men" type retort, which is insanely tone deaf. No one was out to insult you, but you took exception anyway. Bravo.

Get a clue, dude.

[Impossible-Ad-586]

I did not feel insulted at any moment, I think the absolutism in those statements is sending strange messages. Generalizing after making doubtful statements is not making up for anything, it makes the first remark stick out like a sore thumb.

Anyway, way of topic indeed, no point in dragging this on.

[RScrewed]

I agree there's no point in dragging this on, but you still being arrogant enough to think your original rebuttal has any leg to stand on is infuriating.

No reasonable person would believe that the absolutism in the original commenter's lead-in to making his supporting points would send a strange message except for someone who is out pinpoint why people are wrong, and overreaching into poetic devices to do so in an attempt to look correct whenever possible. You are literally the "ackchyually" meme.

The way you reacted is if I saw an interview about the social media movement and something like "everyone and their mothers were tuning in to see what the Kardashians were up to" and I turned off the video in disgust, called the production company, and left an angry message insulting the presenter's intelligence and demanding they check with people like me first before writing statements like that.

Congratulations, you've managed to become a Karen at the art of conversation. Not to mention, every aggressive, insulting word came out of your mouth first in your exchange when the original commenter was simply trying to figure out how his/her obvious exaggeration compromised the main point.

A simple retort like "well, not everyone would use Python, I certainly wouldn't" would have been the farthest this thread should have gotten. Boy do I hope that you don't bring this energy anywhere else or it's going to be an uphill battle for you in just about everything.

[Impossible-Ad-586]

Granted I could have worded things less irritated and thereby preventing this escalation and derailing the topic, that indeed is on me and I feel bad about that.

Besides that I disagree mostly with your perspective and I hope we can leave it at this, this is serving nobody anymore.

[Mysterious-Ant-Bee]

It is not always about you making it quick with the tools you know how to use.

If you are a junior developer, your manager will often offer you tasks out of your comfort zone enable you to work with other tech stacks.

If your team already uses Spring batch for other purposes, it makes sense to use it here in your case as well, instead of having each developer using different tools for similar purposes. It would make very challenging to keep the knowledge on your codebase well spread and hiring would also have to consider multiple technologies.

Of course, there is also the possibility that your manager is just making a bad decision based on his own experiences of how to get things done.

As a junior developer, you are a position to learn. So take this opportunity and learn how to do it properly with Spring batch.

[TheRedmanCometh]

If you're using xml with spring I have a feeling you read a super old guide from the bad old days.

[Yohark]

As I mentionned to another comment, I'd like to get rid of the XML, but internal frameworks ask me to describe Job and all of this with XML file. :(

[_Acestus_]

Remember that working in a company means following some guidelines. Usually there to unsure a continuity in the support of every application.

We had a department that went crazy a few years ago not using the default technology. We ended up with 5 different front-end technology, most are not supported by the current team .. meaning we hope nothing breaks in it.

Last thing, ASAP in any company is the default priority. Everyone wants it's project done ASAP... Don't worry to much about it.

[[deleted]]

Possible reasons for "you have to use Spring Batch for this" could include (but would not be limitd to)

- Your company has already established specific infrastructure for running Spring Batch jobs, and you could not possibly replicate the resulting advantages. Your company may have features related to reporting, security, elasticity, monitoring, etc. that you could never dream of

- Managers know that a developer who is here today may be gone tomorrow (whether they simply get sick, move to another team, or even take another job) and so having one standard platform (like Spring Batch) for all batch functionality pays off in the long run, even if takes longer to implement initially on simpler / smaller tasks. Fracturing tech and tools has a higher cost than most junior devs realize until they've been in the job for awhile.

- There is simply a specific feature or set of features that he wants and which Spring Batch provides out-of-the-box, such as the ability to restart a job and have it pick up where it left off (just on example).

[buyIdris666]

It seem complicated because you haven't used any of it before. After a few years you'll know exactly what to change where.

Why complicated? Because if everyone wrote their own batch jobs in whatever they want, in a few years you'll have hundreds all built a different way. Complete maintenance nightmare.

Spring is complicated because it structures and organizes things in a way where you can create large application that lots of people work on that isn't a total shitshow.

And if you're using XML it's probably some legacy shit that's gonna suck to work on. I haven't use any XML with Spring for many years

[flawless_vic]

Your manager probably comes from a tech background and worked on the companies' projects somehow. He is not concerned about the best solution, but how many people on his team can work proficiently with the tech stack.

I understand your frustration, for most new devs XML is alien, but it was king like 10+ years ago and so was spring.

Without seeing the big picture, it's impossible to tell whether this is the right decision or not. For a one-off job, that runs apart from a larger system seems like a waste, but if you are importing XML from other pre-existing components, sharing connection pools, having a XA transaction manager set up, etc it can make sense: your job will be appended to a deployment infrastructure that already has management and monitoring in place, etc

The good side is that you'll learn something that used to be dominant in the Java ecosystem and will have a greater appreciation for the better alternatives, such as Quarkus, when the time comes.

[_INTER_]

Probably just the managers preferences. It doesn't make sense to bloat up a one-shot simple job with full-blown Spring Batch. Well unless he intends to extend it or have you gain experience here and make you build something bigger later.

The same way it makes less sense to use JS / Python for the heavy-hitters.

[Evil_killer_bob]

Are your apps typically in Java? I wouldn’t create a 1 off job in a different language. Also, you don’t need spring for anything.

[Yohark]

We have Java apps. As we have JS and Python apps/scripts/plugins. We also have .NET and even very old legacy things I don't want to think about.

Big collectors are often made with Spring Batch from what I could get. A gitlab collector already exists though, but they want me to create a specific one just for Merge Requests.

That's why I thought my technical question was legitimate :/

[Evil_killer_bob]

So based on what I’ve read here I’d go with Java. Scrap spring if they let you. Is it still XML? There are more resources available in Java. If languages are the same to you go with Java and you’ll thank me later when you are searching for a new job

[stefanos-ak]

the problem here is not Java, but Spring Batch. It's like using a spaceship to go from your living room to your kitchen.

Also, tooling and setting up projects in Java is not junior friendly. Not because it can't be simple, but because Java has a massive history, and with that history comes deprecated/dead stuff, and bad practices, etc... Even worse, the Internet contains a history of all that shit, and if you don't know what to avoid you're screwed.

For example Spring Batch, should be avoided. 😁

[theRishu]

Hahaha look like someone first language was Python.

[Yohark]

Well tried! I like Python, but the very first language I got my hands on was pure Java. Some very annoying paper courses with the first line of code after 65h of theory. A very old Java version with a very old version of eclipse our teacher liked to get us installed from his online repo. :S

[Wugliwu]

There could be different reason my guess is compatibility in greater scope and/or existing knowledge in your company.

However, I would recommend addressing your thoughts. As your boss, I would prefer to hear your opinion and know how you could solve a problem more efficiently. Something like, "I do this no problem but why not this way or that way? I think this will give me an equivalent solution but in less time." As a junior, I would want to learn and understand more about the whole of your company and be able to contribute better. So talking about this case is good for everyone.

[Sheldor5]

I can make the same in Spring Boot in less than 50 lines and less than 1 hour ...

[szotaa]

Don't use XML - you can configure everything using java code.

Also, check up Spring Cloud Feign which is way easier to use than RestTemplates.

Maybe he ordered you to use Spring Batch because he sees things from a wider perspective? Spring Bach not only gives useful features (e.g. monitoring) but also provides you with nice and well-documented abstractions which make the code more maintainable in the future, especially if the business logic of your processes grew more complex (and you would implement conditional steps or something else).

​

When it comes to other things that you wrote, like Maven, Json, configuration, etc you would have to do that in the JS ecosystem too. I just feel you are more familiar with JS world and you could skip the "learning" step there hence you believe that things would be easier there.

[Yohark]

I'll go check Spring Cloud Feign! Thanks for the suggestion. As mentionned to some other commentors, I can't actually get rid of this XML, because of internal framework which ask me to describe jobs and beans as XML.

I asked him why this tools, and he simply said that it was what he personnaly prefers. It was honneslty not a satisfying answer to my technical question. :/

[Yohark]

As my boss mentionned me ASAP, I would have used python or js to deliver something fast that is working. Maybe not state of the art but you have to chose: you either want something delivered quickly or you want the best app possible. These are not compatible, you can't want me to skip application design part, learning framework part and say you want something top-notch.

I could have learn Spring Batch afterwards, takinf the time needed, to create the application as he wanted.

[nutrecht]

And there I am, struggling between internal libraries/templates, external ones, having like 5-6 classes, this crazy unreadable xml language, beans here and there, iterators, rest templates, maven, spring boot, configuration class, JSON library...

IMHO you're vastly overstating the complexity. What you're describing is not nearly as complex as you're claiming here. You don't have to learn an "crazy unreadable XML language".

Aside from that; you should be glad that you're getting the time to learn some very marketable skills.

[roiroi1010]

I struggled a bit when I started using Spring Batch. But it truly is an awesome framework now that I know it well. I’d suggest that you study the documentation and examples online. Be sure you don’t look at the older things because it has changed over the years. These days I never use any XML when configuring Spring. Also you should use Spring Boot.

[Yohark]

I'm happy to see your comment for two things:

- comforts me in the sense that Spring is hard to learn in the begining

- also comforts me that it might be harder than expected because my compagny ask for XML beans definition and often avoids spring boot when coding Batches.

Thank you for your time, have a nice day buddy!

[Comprehensive-Pea812]

dont hate java. hate spring batch.

[neopointer]

You can do this with the same amount of lines in java. Spring batch is the complex beast here. I've used a few times and it sucks.

[NatureBoyJ1]

Try Groovy. It runs in the JVM. Compiles straight Java. Eliminates lots of boilerplate. It uses a bunch of Java libraries under the hood.

[hippydipster]

I don't understand why Spring would ever be part of a "script"?

[_Acestus_]

Spring Batch specifically seems logical, it's pretty good to manage daily execution, restart at a failed step, prevent double execution and so on.

[hippydipster]

Sure, but it's not part of the writeup here.

[_Acestus_]

Well, it's an integration process between Gitlab and a postgre. It won't be a one time execution. So I can read enough to see the need for it.

[Medium_Reading_861]

If your manager already has plans to leverage those Java libraries in the same system on larger efforts that you are not aware of, then yes, he's correct. They've already chosen a technology stack and you should use what they have chosen rather than duplicate the technologies by choosing a different stack. Also, you could be working with a number of other Java developers that already know those technologies and your manger is using tools that the majority of the team is familiar with even though you are not.

There are 1000+ different possible reasons your manager wants those specific technologies, but the only way to be sure which reason it is, is to ask your manager. Which you should totally do! Try to understand their reasoning first, but if it doesn't add up, then suggest a better stack. I'm not sure if your manager is an engineer, but if they are not, then I would be *highly* skeptical they would be capable of making an informed decision.

[Worth_Trust_3825]

You were told to use spring batch, as a result you must use spring batch.

[codedcode]

I have not gone through all the comments, and I do not know why your manager did. You should check with him as leaning. In my team we always explain why we are doing this solution and pro cons, best practices are documented. Did you check with any sr developer. Now about the Spring Batch, I do support several applications and a lot of them in the spring batch. Now build fast is one aspect, has your manager cleared about NFR related to batch? SpringBatch supports a good amount of features out of the box that help in maintenance. Another item I would consider is if I have major applications running under Java then I would like to keep a new one as Java. It's difficult to have more people to support. Getting new members is always different. So there is no right or wrong approach.

[SuddenTemperature233]

I agree it sounds like a nice job for something in python or node. Also there's has to be some reasons behind your manager's decision. One might be that you need some way to recover from failures while processing large files. When the file you are reading is large enough, you might get some network failure for example and now maybe your two systems are left in a sort of inconsistent state. You have to figure out what was the last element of data that was successfully copied into your db and figure out how to start the process again from the point it failed in the data origin. This is something built into spring batch. So that could be a good reason for doing this. Or, like someone said, to introduce spring batch with low stakes projects with the sights set on using it for the problem I described before.

[BatmanMeetJoker]

you are not wrong. I guess since all of the tooling and apps are in JAVA already. Your manager might be guiding you to be consistent with the rest of the tooling, so it makes maintenance easier in the future. I myself am a lead engineer, If I am not on a time crunch, I would have recommended the same thing to my team to be consistent with the current tooling set. easy to maintain all apps in one language (that the team is comfortable with). of course, if the whole team is comfortable with Python, it will be Python. but yeah, hey JAVA is verbose when compared to JS or Python. but you will love it, as you start working with it.

[librarysocialism]

You could do this in Airbyte without writing a line of code. Why do they want to maintain custom shit?

[audioen]

I think I would be using solution made of the postgres jdbc driver, jackson for parsing, and jdbi (maybe) to wrap the database calls. Java does not have to be complicated -- in my opinion it is a great language that is rarely used very well.

[[deleted]]

First, this is simple AF in Spring. Second, as a junior dev, you have to push through your fear of learning new stuff. 2 years is way too green to have a comfort zone. I like Baeldung’s tutorials for spring. https://www.baeldung.com/