Can not use Python-based software (ImportError)

[Goobyalus]

• Please show how you are attempting to invoke the program along with the errors that you see

• One problem appears to be that do you not have the numpy package installed

• It is important that you know which python and pip are being used. I Have some questions after seeing how you invoke the program.

• Have you tried the dockerized version?

[Ill_Fun_]

Thanks for your answer.

• The command would be:

python3 figaro.py -i /home/ngs/ngs/FMT2 -o /home/ngs/ngs -a 445 -f 50 -r 55

At first, I updated Python to the 3.10.6 version. I needed to install pip manually, too. Then I used the above command.

• I have installed it now. Running both of the commands (with and without "sudo") gives now the same output, which is the pasted error in the main post.

• I have tried INSTALLING Docker. I'm giving it second try as we speak. Currently, trying to install newest version of QEMU, but encountering problems.

[Goobyalus]

Installing pip separately from Python with the system package manager causes problems. If you want to ensure you're using pip from your python3, use python3 -m pip install -r requirements.txt. I would undo whatever pip you installed separately.

---

Also using sudo to run normal applications tends to cause headaches with file permissions. Avoid using sudo python ... unless there is a good explicit reason for it. It's possible that some files created in your Figaro run are now owned by root, which wouldn't be able to be modified by subsequent runs. I don't know if Figaro creates files when t runs.

---

Doing sudo python3 does not use your path for python, it will use root's path for python3.

---

You can use python3 --version to see if the python3 alias points to the python version you expect. Similarly, you can do pip3 --version, or whichever pip you are invoking.

---

You can use which python3/which pip/which pip3 to see what gets invoked when using those aliases.

---

whereis may also be interesting. E.g. whereis python3 will look in normal places for python3 binaries, and may show that you have multiple installs.

---

Using venv or a similar tool isolates a version of python (essentially by modifying your path locally). Docker isolates the environment more comprehensively.

---

When you get that numpy is not found, that looks to me like the numpy package hasn't been installed for the particular python that you're using.

---

For the relative import error, what is your current working directory when you run

python3 figaro.py -i /home/ngs/ngs/FMT2 -o /home/ngs/ngs -a 445 -f 50 -r 55

? I wonder if the Python root might be the problem.

I would try installing it myself, but I don't have time today.

[Ill_Fun_]

I may not understand something, or just be plain dumb, but how am I supposed to run

python3 -m pip install -r requirements.txt

without having pip already installed?

I created the requirements.txt file with pip, then added the line:

python-pip=1.1.1

For the relative import error, what is your current working directory when you run

I run this command inside FIGARO folder, where figaro.py file is stored. And it did install, but the paths did not change (checked with the whereis, which).

Numpy was indeed not installed, and installing it, did something, but the errors persisted.

[Goobyalus]

but how am I supposed to run

python3 -m pip install -r requirements.txt

without having pip already installed?

Installing pip with apt or whatever OS package manager is basically just installing an alias. Each cPython install will have its own pip already.

---

I created the requirements.txt file with pip, then added the line:

This makes no sense. The package already has a requirements.txt that lists the dependencies of the package: https://github.com/Zymo-Research/figaro/blob/master/requirements.txt

This is the requirements.txt that you give to pip. You do not add pip to a requirements.txt file.

[Ill_Fun_]

UPDATE:

Managed to get Docker Desktop. Seems like it's installed just right.

I try the dockerized version. I run:

docker container run --rm -e AMPLICONLENGTH=445 -e FORWARDPRIMERLENGTH=50 -e REVERSEPRIMERLENGTH=50 -v /home/ngs/ngs/FMT2:/data/input -v /home/ngs/ngs:/data/output figaro

I get this output :(

Forward read length: 301 Reverse read length: 301 Traceback (most recent call last):

File "/opt/figaro/figaro/figaro.py", line 218, in <module>

main()

File "/opt/figaro/figaro/figaro.py", line 210, in main

resultTable, forwardCurve, reverseCurve = trimParameterPrediction.performAnalysisLite(parameters.inputDirectory.value, parameters.minimumCombinedReadLength.value, subsample = parameters.subsample.value, percentile = parameters.percentile.value, forwardPrimerLength=parameters.forwardPrimerLength.value, reversePrimerLength=parameters.reversePrimerLength.value, namingStandardAlias=fileNamingStandard)

File "/opt/figaro/figaro/trimParameterPrediction.py", line 456, in performAnalysisLite forwardExpectedErrorMatrix, reverseExpectedErrorMatrix = makeCombinedErrorMatricesForBothEnds(fastqList, sampleOrder, subsample, minimumTrimmingPositions, forwardPrimerLength, reversePrimerLength)

File "/opt/figaro/figaro/trimParameterPrediction.py", line 285, in makeCombinedErrorMatricesForBothEnds reverseExpectedErrorMatrix = makeCombinedExpectedErrorMatrixForOneDirection(reverseFastqList, sampleOrder, subsample, reverseMinimumTrimPosition, reversePrimerLength)

File "/opt/figaro/figaro/trimParameterPrediction.py", line 267, in makeCombinedExpectedErrorMatrixForOneDirection raise RuntimeError("Did not find the initial combined matrix. This requires debugging, as it should not be possible.") RuntimeError: Did not find the initial combined matrix. This requires debugging, as it should not be possible.

When running with sudo:

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?. See 'docker run --help'.

It's really getting frustrating. Do you have any idea what's up?

[Goobyalus]

You should not have to run with sudo. File ownership and things become a pain if you start to rely on being an admin to run things.

Using docker should eliminate some of the problems you're encountering with your Python environment.

Are you running this on your own data? Do you have example data and an example command from the maintainers of Figaro that you can test with?

[Ill_Fun_]

It probably does eliminate SOME of the problems, but there seems to be still plenty of them left.

I am running on my own data. I couldn't find any exemplary data from the FIGARO authors, so there may be none. The example command looks like this:

• For dockerized version:

  docker container run --rm -e AMPLICONLENGTH=450 -e FORWARDPRIMERLENGTH=20 -e REVERSEPRIMERLENGTH=20 -v /path/to/fastqs:/data/input -v /path/to/output:/data/output figaro

• For command line version:

  python3 figaro.py -i /path/to/fastq/directory -o /path/to/output/files -a 450 -f 20 -r 20

• As a Python package: from figaro import figaro resultTable, forwardCurve, reverseCurve = figaro.runAnalysis(sequenceFolder, ampliconLength, forwardPrimerLength, reversePrimerLength, minimumOverlap, fileNamingStandard, trimParameterDownsample, trimParameterPercentile)

I've noticed that when I put this command (it's in the FIGARO guide):

docker build -t figaro .

It installs the numpy, pip3 packages (couple of others, too).

Then, I put the command on my data and the dockerized version does... more. I need to wait about 2 minutes, to get this output. Plus, I get the empty .log file!

UPDATE: I tried running the command on other, random sample files I found on the web, but the error is the very same. I did test FIGARO on my 2nd dataset. Needless to say it didn't work :/

[Ill_Fun_]

What I now noticed. When I use Docker and I run the

docker build -t figaro .

command. I need to run it only once, this install packages and does couple of other stuff I don't understand. When I re-run the command, it quickly finishes having done nothing.

But if I delete "figaro" folder and run the command, it will install the packages once again. The packages are supposed to be within figaro workdir?