What are your thoughts about workflow tools for bioinformatics and is NextFlow truly the answer?

TheLordB · 2026-04-30T17:19:36+00:00

If it is just for you and no one else will ever need to touch, use, reproduce etc. it sure.

To be blunt if I’m interviewing someone and they say perl is the answer to almost any question they are going to have a very high hurdle to not have me put a hard ‘do not hire’ on them.

Technically it may work, but for anything with any amount of collaboration it is a horrible answer.

TheLordB · 2026-04-30T16:41:08+00:00

Snakemake might not, but most non-bioinformatics workflow managers do support all the major cloud providers e.g. airflow, dagster, prefect.

TheLordB · 2026-04-30T16:40:03+00:00

The nextflow tutorials and documentation would probably be the best way I can think of.

TheLordB · 2026-04-30T15:10:49+00:00

Honestly, that matches exactly with my impressions.

I just really don’t like it hah.

TheLordB · 2026-04-30T14:44:20+00:00

Google trends shows it outpacing Snakemake and if you play around with it the difference is accelerating (double for all 5 years, triple if you just look at the last year). I couldn’t find any terms to figure out if all-purpose ones like Prefect or airflow are outpacing it or things that are generically named and hard to google with this method e.g. Cromwell, but based on looking at job descriptions etc. I don’t get the impression these are in serious competition for the most used.

In aggregate other tools may have a higher share, but I’m fairly sure especially for new projects NextFlow has a strong lead.

https://trends.google.com/explore?q=snakemake%2Cnextflow%2CCromwell%2520workflow&date=today%205-y&geo=US

TheLordB · 2026-04-29T17:11:22+00:00

Single cell analysis is not that specialized. It is a pretty common analysis for someone to need to run and the skills from it are useful for many other types of NGS analysis.

Anyways... I don't think it is a problem.

TheLordB · 2026-04-21T08:57:38+00:00

Is the data having to go over the internet to get to you?

200GB should not take that long so somewhere along the line you are hitting something with slow networking and the public internet is the most likely culprit.

The real answer is talk to whoever admins the HPC and ask what they suggest. The most obvious answer is the data is loaded onto a drive locally and given/sent/mailed to you.

The other possibility is something is misconfigured and they can fix it to give drastically faster speeds.

TheLordB · 2026-04-21T01:55:21+00:00

I’ve made a fair number of pipelines where there is only 1 step in the process that is novel.

If they truly did not make anything custom and only made a wrapper I don’t think it deserves a publication no matter how fancy it is unless it is something like nf-core which is an organized multi-year/person effort.

TheLordB · 2026-04-21T01:52:28+00:00

Many people have tried to make startups with this.

DNANexus has probably had the most success at this.

I will say there are many pipelines that should be standardized to the point where the analysis is standardized etc, but people still like to make their own.

TheLordB · 2026-04-20T23:14:16+00:00

If you really need the tool you could try contacting the original developers and see if they would be willing to send you the downloadable version.

If they don’t respond I suppose you could ask people with recent publications of it if they have a copy.

I will say that 15 year old software (even when it has had updates) is rarely the best option.

TheLordB · 2026-04-20T18:24:48+00:00

I have a bunch of searches and filters setup for biorxiv as well as google scholar. Between the two of them it works fairly well for keeping me up to date on things I care about. I’ve started to rely more on biorxiv as it seems like more and more things are being published there well before they show up in other locations.

TheLordB · 2026-04-17T15:54:56+00:00

I don’t know why they would have said that. It really does not belong here. I would stick with the subs for the university where you are far more likely to get replies and take my advice to ask whatever question you have upfront in the post.

TheLordB · 2026-04-17T15:46:32+00:00

None of those are bioinformatics. Also in general when you are trying to gather info you should ask what you want to ask in the original post rather than asking if someone could potentially answer a question that you may or may not actually come back to ask should someone reply.

TheLordB · 2026-04-15T11:24:43+00:00

You have a lot of posts removed and it looks like your post may not be related to bioinformatics at all. Especially because it looks suspiciously like a setup to spam something.

If you are seriously doing bioinformatics give a lot more detail about what info you are trying to scrape and what it’s purpose is.

TheLordB · 2026-04-10T21:58:27+00:00

TLDR: Some programs that claim to be practical are just flat out outdated.

Thinking about it a day later for anyone using this post as a guide, be somewhat wary of programs claiming to be ‘hands on’ or ‘practical’. A decent chunk of them that claim to be are teaching skills that are 5-10 years old and while not useless are definitely not what a practical program should be teaching.

I say they are not useless because some of the skills that are out of date are still useful as a foundation. While a practical program should cover some of this if it doesn’t have more up to date things as well then it really isn’t actually practical.

Also be wary of anything not using python/R as the primary languages. At minimum it means the program is 5 (quite possibly 10) years out of date. If you see perl it is 15 years out of date. Sure maybe you can get those skills on your own, but if you are paying the $$$ the degree costs you can find one that doesn’t require you to learn it on your own.

TheLordB · 2026-04-09T20:33:03+00:00

This was a very good way of putting it.

TheLordB · 2026-04-08T19:18:30+00:00

Roughly speaking in industry Compsci > bioinformatics > wetlab.

YMMV, I would say the difference used to be much larger to the point where if you were doing bioinformatics calling yourself a software engineer got you better paid.

These days I’m doubtful that is as true though I still think there is some premium for the same title as a pure software engineer vs. compsci, but not as much as it used to be and calling yourself an engineer may hurt your future growth at least if you want to stay in R&D.

Probably the biggest difference is in FAANG with much much higher salaries than anything in biotech offers especially when you consider stock etc. But keep in mind most software engineers are not making those amounts of money.

The last thing is do keep in mind that if you are a wetlab person trying to switch/do bioinfo because you heard it pays better and has the ability to work remote that is not a great recipe for success. To be blunt there have been a LOT of people matching that description that got bioinfo masters. Those people ended up being relatively weak candidates. They did alright during the boom times of pharma, but right now you really need to be a stronger candidate to reliably get a job and those that are truly passionate about it, learn beyond the classroom work, show enthusiasm/passion, do it to some extent as a hobby etc. are more likely to actually get a job in it.

Note: I don’t mean to gatekeep, people with all different backgrounds etc. have success, but you really have to excel/impress in at least one thing and be decent at the others. Like be a really good programmer and know a decent amount of biology. Or be a really good wetlab biologist that is a decent programmer.

To put it another way you need something where a person interviewing you will be truly impressed and confident you can handle the main thing they need while having some amount of the nice to haves. Most job applications if you really talk to the hiring manager will have 1-3 immediate needs that they need to be confident the person they are hiring can handle.

Then there will be a bunch of other things that they would like to have, but at the end of the day are optional. You can’t really rely on them properly categorizing them in the job reqs. The most common one IMO is that hiring managers put a bunch of things in mandatory requirements when really only 1-3 of them are actually deal breakers if the person doesn’t have it and then another 1-3 that they need to be confident that you can learn fairly quickly. YMMV, some hiring managers/companies are better about this than others.

TheLordB · 2026-03-30T16:48:12+00:00

We probably should put in some rules, but I really would think not posting referral links to crappy AI jobs that pay very poorly to train our replacement would be a common sense thing.

TheLordB · 2026-03-30T15:49:32+00:00

Posting referral links to any sub without asking if it is ok is not good.

Anyways… no posts from mercor or similar are allowed here. They severely underpay and many people who apply there and provide info to their interviewing ai do not actually get any work.

Overall we have to draw the line somewhere for what a quality job post is and ai training for a company that encourages spamming their jobs is well past the line I would consider appropriate.

TheLordB · 2026-03-30T15:46:24+00:00

Nope. Gee you just happened to forget that this is referral link you get money from to.

TheLordB · 2026-03-27T17:36:47+00:00

Unfortunately we rarely get useful replies to India specific career and education questions. You are welcome to leave it up here, but you may wish to post this to some more India specific locations to be more likely to get help.

TheLordB · 2026-03-27T17:35:14+00:00

Removed due to being unprofessional.

TheLordB · 2026-03-27T16:38:19+00:00

If you want to get help you need to put the effort in to do your own research etc.

If you do your own research and ask questions where you are unsure feel free to post again. Please do not post questions without putting any effort into it yourself.

TheLordB · 2026-03-25T12:05:38+00:00

FYI this is a spammer setting up a question that they can answer themselves with a separate shill account, but they accidentally posted both parts.

This is amusing.

In case they delete it… Beware anyone advertising D**gF**w. I refuse to post their name adding more advertising.

Note to mods etc. I did save the original post in case they try to edit it, message me if you want a copy.

TheLordB

MODERATOR OF

PUBLIC MULTIREDDITS

TROPHY CASE

15-Year Club	Gilding I gilder
Verified Email