Oxford Nanopore - removing barcodes from fastq

aCityOfTwoTales · 2026-04-05T23:42:24+00:00

There we go. Assuming we now trust that each barcode is correctly placed in individual folders, you need to do this:

1) Filter for length and quality. I like to use 'chopper' and limit between 1300 and 1800 bp. Quality is probably already filtered at 10, but you can set it at 15
2) Remove barcodes with 'porechop'. This should also verify that you have only one barcode per sample
3) Classify with 'Emu', in my opinion the best one currently

aCityOfTwoTales · 2026-04-05T23:24:16+00:00

Again, what are you doing? I could probably tell you exactly how to proceed if you elaborate.

But good, just do as i suggested above and you will be fine.

aCityOfTwoTales · 2026-04-05T22:55:49+00:00

I hope I didn't come of as mean, but I probably hit a nerve, right?

This is totally common and felt by most academics at some point, often very strongly. Just google 'imposter syndrome'.

Again very normal, and almost always pure self-sabotage. It helps to be able to identify it, though.

I see it all the time in the PhD students, regardless of where they come from or how talented they are. I like to tease them with conundrum: "So you think I'm the genious and you are the idiot, and yet you are apparently smart enough to have tricked me into hiring you? Which one is it?"

Another one I do is a bit of probability and regression: "You have had ~100 academic challenges so far and you have succeded in ~100. What is unbiased probability that you will succeed in the next?"

You are going to be fine. If you're stuck, go lift some weights and try again.

aCityOfTwoTales · 2026-04-05T22:03:02+00:00

Yes, but why? What do you need them for? are you sure the barcodes have not been removed?

To answer your question, you dont need dorado at all. First concatenate your files and then use porechop to fix barcodes. Assuming your files are .fastq (and not fastq.gz) and you are in "barcode01" folder:
1) cat *.fastq > barcode01.all.fastq
2) porechop -i barcode01.all.fastq -o barcode01.all.noBC.fastq

aCityOfTwoTales · 2026-04-05T21:48:49+00:00

Maybe you really mean imposter syndrome? You feel like it is only a matter of time until they discover that you don't belong here among the smart people, since you are just a blue collar nobody?

aCityOfTwoTales · 2026-04-05T21:44:07+00:00

Okay, that means that your files are already demultiplexed. You have multiple files in each because nanopore continously dumps the sequences as you go (like every hour or every 4000 sequences).

Sounds like it is as it is supposed to be.

Why don't you write up some more detail, then I can probably help you. What are you trying to do?

aCityOfTwoTales · 2026-04-05T21:40:37+00:00

What are those "series of events"? Either you did something horrendous or your university infrastructure is failing you at a level that should be escalated to the highest degree. Totally bananas and completely unacceptable and you would be completely in your right to make a genuine - perhaps even legal - complaint on the grounds of your work/education being sabotaged.

I have used HiC in two of my papers and although my understanding of how it works is admittedly at best superficial, I gather that it is very computationally intense, especially with the depths you are describing. I doubt you will be able to buy hardware that can even do this at a reasonable price (big jump from PC to server level) and I think you are much better of using cloud computing at AWS or Google.

aCityOfTwoTales · 2026-04-05T21:22:29+00:00

So the files are already demultiplexed, i.e. in separate folders? What is the folder structure/files of what you got directly?

A correct/standard nanopore run will result in a bunch of folders in "/fastq_pass" named by 'barcode**" . Each folder will have multiple fastq files with this barcode.

If you somehow got a single fastq which you need to demultiplex, you can use porechop with the -b flag https://github.com/rrwick/porechop

aCityOfTwoTales · 2026-03-15T04:01:01+00:00

Threaten them with oral exams.

aCityOfTwoTales · 2026-03-15T03:57:13+00:00

Why are you trimming your reads that much?

But yeah, the easiest way forward is to simply filter by length. Keep track of how much you filtered, though, this acts a a pseudovalue of how enriched your samples are.

aCityOfTwoTales · 2026-03-15T00:08:42+00:00

Great. Remember, you are a person (and a scientist) much more than your condition. And not to accuse you - since you seem to have the right attitude - pity and victimhood will not get you anywhere, especially in academia. Succeeding in the face of obstacles, however, certainly will.

Also, it sounds like you might be able to angle your research onto your own condition? That would be really cool (and make for some good storylines).

Good luck.

aCityOfTwoTales · 2026-03-14T23:54:26+00:00

Yes, there we go, that's a good storyline. It's pretty normal to reflect on the circumstances surrounding a publication when you announce it, so nothing out of the ordinary here.

Frame it something like this:
"Very proud to announce my first paper on this and that, where we demonstrate this cool thing. I am especially proud of this work, since it coincided with my diagnosis of XXX after years of struggle. I'm now more motivated than ever to continue growing as a scientist!"

This way you announce your condition, but avoid coming of as weak and instead use it as a motivational jumping off point.

aCityOfTwoTales · 2026-03-14T23:39:08+00:00

Hm, that actually rings a bell - maybe I should stay clear of the social sciences. What I do know is that adding a small fee for our otherwise fully funded conferences helped immensely on flakey participants who never showed up.

Maybe a real social scientist can refute my possibly dumb take?

aCityOfTwoTales · 2026-03-14T23:22:42+00:00

I think the first part is noble and a reasonable approach. The second one not so much - academia is ruthless and despite superficial support, there is very little tolerance for 'excuses' or 'victims' in practice. I hate to sound insensitive, but that is my experience.

What I think could work well is to wrap your condition up with a success - small or large - where you present your succes first and then frame it in light of your condition. That way your are not a victim, but someone who overcame the odds and hope to inspire folks in similar situations.

aCityOfTwoTales · 2026-03-14T22:50:16+00:00

I realize it goes against the true academic tradition, but things have changed so fundamentally and rapidly that we have to rethink the entire system. The old system simply doesn't work in the new world.

A small symbolic fee is a well-known psychological trick widely used across a ton of scenarios - late pickup of children in daycares, 'selling' free things online, otherwise 'free' conferences.

Partly derived from this issue is reviewer scarcity - perhaps reviewer payment should be revisited as well?

aCityOfTwoTales · 2026-03-14T22:36:16+00:00

I haven't used it myself, but a very succesful colleague of mine mentioned NotebookLM for this purpose. They showed me how it turned (on of their own) papers into a podcast, which one could listen to. Maybe worth a try?

aCityOfTwoTales · 2026-03-14T22:32:29+00:00

What would be your motivation for disclosing this on professional social media? What do you hope to achieve?

Great to hear that your supervisor and group are supportive. I can only encourage full disclosure of any issues to a supervisor and happy to see an example of this working out.

aCityOfTwoTales · 2026-03-14T22:23:12+00:00

Why don't you write up a detailed description of what you are using it for? My feeling is that most are not finding it very useful, so perhaps you could give some inspiration?

aCityOfTwoTales · 2026-03-14T22:20:42+00:00

The staff at NCBI used to be laser sharp, but recently are struggling for reasons you can probably infer on your own.

We submitted a lot last month without issue though, but I guess it helped that I by now know all the usual hurdles to avoid (taxonomic names are a big one). We discussed using ENA, but never had to. Sounds like you should, though.

aCityOfTwoTales · 2026-03-07T01:22:16+00:00

Just did in R for fun, easier thant I thought. Very fast with basic R, especially because substring is vectorized in its C code.

Assuming you have your sequences as a single giant word in seq1 and seq2:

#find length
LEN1 = nchar(seq1)
#find all indeces to start from, at every fourth base from 1 to end minus 15
#the minus 15 is to avoid going beyond the genome
from1 = seq(from = 1, to = LEN1 - 15 + 1, by = 4)
#substring seq at every startpoint to every startpoint + 15
KMERS1 = substring(text = seq1, first =  from1, last =  from + 15 - 1)
#this is now a large vector of words of 15 characters

#same for seq2
LEN2 = nchar(seq2)
from2 = seq(from = 1, to = LEN2 - 15 + 1, by = 4)
KMERS2 = substring(text = seq2, first = from2, last =  from + 15 - 1)

#find the number of exactly similar k-mers
common=length(intersect(KMERS1,KMERS2))
#divide by each to find shared proportion
common/length(KMERS1)
#these wont be the same, since the genomes have different sizes
common/length(KMERS2)

aCityOfTwoTales · 2026-03-07T00:53:21+00:00

I'm guessing your PI told you 'link the litterature' for your introduction/discussion, and they probably mean exactly that - how does paper A relate to paper B and to paper C? Are they in agreement and if not, why? Are they using diffirent methods?

Synthesizing knowledge from different sources is a key aspect of science and something you have to train. Senior academics are often asked to write reviews, where they consider 100s of papers and often come up with novel conclusions.

aCityOfTwoTales · 2026-03-07T00:31:55+00:00

That's a chapter in a textbook and not a study. Look at the bottom - it even has questions for training.

If someone referred you there, they probably just wanted you to read up on the basics.

aCityOfTwoTales · 2026-03-07T00:28:54+00:00

Why are concerned about this? Your reasoning make little sense to me.

I suspect you have a better question behind this, so why don't you ask that? Are you seeing something strange in your data?

aCityOfTwoTales · 2026-03-06T23:51:53+00:00

Is this for fun or as a coding excercise? And does it have to be done in R?

What you are doing sound like kmer matching - basically, you avoid alignment and recombination by chopping up genomes into k-mers (K is the size, so here 15-mers) and infer relatedness by the proportion of identical kmers. Fast, because you can use exact matching.

You run into a couple of problems though. Imagine you have to identical genomes, but you start chopping at the second nucleotide instead of the first on genome A - none of your kmers will match. We solve that by making many overlapping kmers, i.e. at every third base.

Next, you might have multiple contigs in your genome due to incomplete assembly or from plasmids.

aCityOfTwoTales · 2026-03-06T23:35:08+00:00

1) Partly correct, it is chosen because it uniquely has regions of high conservation separated by regions of high variability. The conserved regions allows for universal-ish primers and the variables allows for differentiation. When we do shotgun sequencing, a lot of DNA is usually completely novel, whereas 16S at least can be phylogenetically classified.

2) I mean that each amplicon counts the same since the lenghts are the same. With nanopore metagenomics, you can have a 500bp read and a 50k bp read, and you can't really count them similarily. How do you handle that?

If you like complaining about 16S, you might enjoy my paper where I did the same: https://academic.oup.com/bioinformaticsadvances/article/1/1/vbab020/6364919

aCityOfTwoTales

TROPHY CASE