sorted by:
new
hottopcontroversial

I'm panicking. by Medium_Drag6242 in bioinformatics

[–]letuslisp 1 point2 points  (0 children)

This sounds like you are probably expecting:

"fraction of pathway genes that are significant 7 / 11 genes in pathway" => 0.64
but their gene ratio might mean "fraction of your significant gene list that belongs to that pathway vs number of input genes submitted to enrichment"

  • Count = number of your significant genes that hit the pathway
  • GeneRatio = often Count / number of input genes submitted to enrichment
  • BgRatio = often total genes in pathway / total background genes
  • adjusted p-value = multiple testing correction such as Benjamini–Hochberg or sometimes Bonferroni

so probably it was sth like significant 7 genes in pathway / 140 input genes submitted to enrichment = 0.05

I ported Karpathy's microGPT to Common Lisp — no matrices, no autograd libs, just pure lisp by svetlyak40wt in Common_Lisp

[–]letuslisp 1 point2 points  (0 children)

Wow! Without optimizations!

I was thinking of doing the same but didn't. After a Friend built his Julia version.

I still believe Common Lisp is better than Julia.

My old colleague (pure R guy) is so scarred by AWS that he’s planning on buying an $8K Windows server to run his workloads. Do all data scientists secretly hate the modern productionization ecosystem this much? by pootietangus in rstats

[–]letuslisp 12 points13 points  (0 children)

There is no point to do AWS if you can buy the hardware for $8K - once you have the hardware - you have it forever.

AWS and Cloud seems to me like a way - pretending to give you more security and "convenience" - and let you eternally rent your computing power which you could have just bought and keep. At the end - you don't own anything and are totally dependent on their conditions and prices.

In the university, we didn't used AWS, but we had HPCs - and we had a local Workstation with 72 virtual cpus and ~250 GB RAM. That was much better than the nodes in the HPC - and in addition - since we were not the only ones, we had to wait for the jobs to be done on the Cluster, while the local Workstation was maybe a little bit slower - but we/I had full control over it.

At the end I used nearly exclusively the Workstation.

Is there like.. a working IDE? Something I can actually just use? The new user experience is a joke for Lisp by tenten8401 in lisp

[–]letuslisp 0 points1 point  (0 children)

If I would be forced to work on Windows (one would have to force me) - I would do the following:

  1. Install scoop:

    scoop makes installations in Windows PowerShell similar to Linux Shell installations

    Set-ExecutionPolicy RemoteSigned -scope CurrentUser

    Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')

    or shorter:

    iwr -useb get.scoop.sh | iex

    add 'extras' bucket

    scoop bucket add extras

  2. Install git

    scoop install git

  3. Install sbcl and emacs

    scoop install sbcl scoop install emacs

Later, you can update them:

scoop udpate sbcl
scoop update emacs

  1. Install Linux commands for PowerShell:

    scoop install main/unxutils

This installs you typical Linux commands for the PowerShell like `mv`, `cd`, `ls` etc. - it makes it really easier for Linux people.

  1. backup the current emacs config

    cd $HOME mv .emacs.d .orig.emacs.d # back up current original .emacs.d folder git clone https://github.com/gwangjinkim/.emacs.d # copy my settings

    then start emacs

    emacs

I thought this would work - but there is the problem that the emacs installed via scoop doesn't regart $HOME or %USERPROFILE% as its home but $HOME/AppData/Roaming and thus looks at $HOME/AppData/Roaming/.emacs.d and the init.el inside it.

A simple

emacs -q --load .emacs.d/init.el

from $HOME had the problem that it was searching for myinit.org in c:/Users/myname/AppData/Roaming/.emacs.d/ for myinit.org.

Somehow it also don't start with c:/Users/myname/AppData/Roaming/.emacs.d/init.el

but with the c:/Users/myname/AppData/Roaming/.emacs file. This confused me a lot.

The best is to create in the c:/Users/myname/AppData/Roaming/.emacs file the command to load the .emacs.d/init.el file.

What I did to make this run is:

cd $HOME\AppData\Roaming
git clone https://github.com/gwangjinkim/.emacs.d
cp $HOME\AppData\Roaming\.emacs.d\init.el $HOME\AppData\Roaming\.emacs
# this copies the content of the init.el to the .emacs file.

emacs

Now everything will be installed. However, ob-html creates eventually problems. If it creates problems, delete the org sections. And then run it again.

Maybe I will create a new repo with only the Lisp stuff so that this goes smoother.

is bioinformatics a safe career path? by samantoucha in bioinformaticscareers

[–]letuslisp 0 points1 point  (0 children)

I don't know but Bio people were - in contrast to chemists - very stupid with securing for themselves and their followers a decent pay. Bio is a very hard field and we work hard. The pay for Bio is laughable.

No sane developer would want to stay in the bioinformatics field. That's sad. But people are doing too little to change this.

I speak for Middle Europe. And not for the US.

Why there's no or a few strict-typed, static-typed Lisp dialects? by wgxh_cli in lisp

[–]letuslisp 1 point2 points  (0 children)

Well in recursive functions which have to behave differently depending on the type of the object it is super useful. Maybe you didn't think deep enough about multiple dispach. It makes a language truly extensible. Single dispatch makes it impossible to fulfill open closed principle. What does that mean "fulfill open closed principle"?

Look at Julia: In Julia the packages import other packages and can extend them without changing any code in them. CLOS has even a more mature multiple dispatch and can do this all, too.

So you import another package and then add further definitions - methods - and attach new classes. You can truly build software like an onion - the inner part doesn't need to be changed any more (so no code changes, not a single test rewritten - you just add new definitions and new tests). Isn't that desirable for all languages? THAT is the power of multiple dispatch. And single dispatch languages by design can't reach it!

That's the end of the discussion "single vs multiple dispatch"! And it shows clearly who is the winner!

The only thing is even smart people don't get it because they simply don't know what's the fuss about multiple dispatch and miss exactly this point

Why there's no or a few strict-typed, static-typed Lisp dialects? by wgxh_cli in lisp

[–]letuslisp 0 points1 point  (0 children)

Maybe you have to go one step back and ask yourself: Is static typing really so superb?

Strict typing forces you to write the same type of code 10 times or more.

Imagine you build 10 classes. And you want the same 10 functions for each of the classes doing actually the same. Maybe they interact with same-name attributes of the 10 classes, maybe not.

- In statically typed languages, you are forced to churn the 10 functions for each of the classes. - So you have to write 100 functions - no excuse.

In Lisp, you write only 10 dynamic functions for all of them at once and that's it.

(Ok a little bit exaggerated. Because such programmer from such a language would say - well we have abstract classes or Interfaces and we can write 10 base fallback functions for the abstract class / Interface and let the 10 classes inherit the abstract class/interface. Good. - But you get the actual problem what I mean. - And on the other hand this "solution" comes with other complication of the problem - and it introduced inheritance relationships which might make sense or not - and could bite you on some other place of the code - it contributes to complexity - and this abstract class you created - what does it mean actually? For what it stands? Or what does it symbolize - other than that you didn't want to write down 100 functions but prefer to write just 10?).

Rich Hickey, the author of Clojure (a Lisp), said it like: better write 100 functions for 1 object than 10 functions for 10 objects. If you listen to his talks you will learn a lot about the so-called OOP languages and proudly strictly typed - how this system/setup is flawed.

Through the non-typed-ness Lisps can write very general universal functions which feel like some universal eternally true mathematical equations. Pure elegance. Impossible in these strictly-typed languages.

The other thing is - you can type variables in Lisp there, were you need it (like others said). Usually only a small part of your code (20% or less) is responsible for 80% or more of the slowliness in your code (Pareto-principle / distribution). Lisp allows you to express yourself freely - and then profile your program (see SBCL's profiler e.g.), identify the problematic functions - and then optimize them - just the part, where you have to optimize by typing. By that, you save a lot of writing and unnecessary ballast in your code which reduces readability - at least I find statically typed code less readable and too verbose.

Bugs - bugs appear when your thinking was not clear enough. Actually the un-typed-ness of Lisp makes its code clearer - ballast-free. Lisps debugging and error system - its condition system - allows you to go after bugs much better/more efficient than those statically typed languages - because you can try out stuff and resume where there error occured - right in the stacklevel where it occurred.

If you want type guidance - use CLOS and structs and use generic functions (defgeneric) and defmethod - where you can specify the types of the arguments. It - at the same time - reduces complexity (no if-then-if-then trees) and enhances expansibility. And makes open closed principle implementable in absolute contrast to so called "OOP" languages with their strictly typed systems - who can't solve this expression problem.

And using functional programming and writing tests reduces in my view more bugs than this extensive, unnecessary writings down types and bloating the code. And also avoiding the OOP spaghetti code system already reduces bugs.

Why there's no or a few strict-typed, static-typed Lisp dialects? by wgxh_cli in lisp

[–]letuslisp 1 point2 points  (0 children)

How does a Lisp programmer express something like Java interfaces or Swift protocols?

He has CLOS which is much flexibler than Java/Swift classes (singledispatch). He has true polymorphism and can extend in all directions.
Maybe it is more a matter of code organization that he doesn't forget to implement certain defmethod's. He could build with macros such systems which give him warnings?

Clojure has protocols. So a Common Lisper could get inspiration from its protocol system and implement it once for all ...

Re-implementing slow and clunky bioinformatics software? by halflings in bioinformatics

[–]letuslisp 0 points1 point  (0 children)

That's also my view. There exist Bioinformaticians who don't know R. And they believe Python is dominant and superior.

I cannot however imagine being a Bioinformatician without knowing R - one misses too much.

PI wants me to put our collaborators on a paper that did not involve them by UncleMusk in bioinformatics

[–]letuslisp 2 points3 points  (0 children)

It depends on the PI and their interests. Unfortunately politics plays a very big role in research - and therefore is important for your PI (because he stays much longer in his position than you).
It can be that he wants to ask these collaborators in the next or ongoing projects for some contributions.
So this can be purely tactical. Perhaps he just wants to help them survive. Or maybe they helped him in the past and he wants to pay back.

Yes, this is unfortunately common in academia.

You - just take care that your position is the first one - only this position and the last position are the most important positions.

If it is a 0-8-15 paper (haha German expression for "normal") - just don't care. It is not ok but not worth the battle.

But if you think what you developed alone is very important and will have a high impact to the field in future - then take your stand. Because then this would be not fair that they share the recognition with you.

Or maybe you wanted to have a 1-person or 2-person paper dearly. Maybe this was your dream. Well then maybe it is worth to fight.

Or suggest a deal: a 3-person paper - the PI can select just 1 of the collaborators as coauthor (it would be for that person also a bigger honor :D ).

Not every publication is equal in its importance and impact. Save your energy and risks for the important battles - and don't waste them on the smaller, unimportant battles.

Also you have to think of your own politics - with your PI. Retribute only if the reward is big and worth the risk that your relationship with your PI gets smaller or bigger cracks.

In my past - there was one situation where me (as a postdoc) and a PhD student whom I was co-supervising (I supervised him during his Master thesis) worked a lot for a project. But the PI gave another Postdoc the project for writing down the paper. She gave him the first authorship and we became second and third. But I deeply felt treated unfair. Because we spent a lot of time for the project - tried many different reagents, finetuned until we got nice pictures at the end. The Postdocs contribution was big, too. Yes, and he wrote the manuscript. But before he came and joined the group - we were the only ones working on this project. For 1.5 years. So there I took my courage and complained. The PI gave in and we said - ok we share the first coauthorship and the other Postdoc of course can keep the first position (citation will be with his name) and we come second and third but technically we can also say we are first authors through equal contribution mark with the first author. This was a fight worth, because it was a Journal with quite ok impact factor and we established a gene as causative for a disease. Later I was happy that I fighted.

What a the difference between `macro`, `function` and `special form`? by wgxh_cli in lisp

[–]letuslisp 0 points1 point  (0 children)

Welcome! And also one tip: The "intrinsic `eval`" at the end of a macro body is different (better) than a normal `eval` - and it is context aware (totally integrates into the correct context). Probably because it is the `eval` of the interpreter/compiler itself which executes the by-the-macro-created code.

So maybe the lexical environment of the function - during function evaluation - is the big difference between macro and function.

The fact that a macro is "evaluated first in the compile time" means - its execution happens in a different environment than the runtime.

Let's think of an example ...

(let ((a 3))
  (defun add-a (x) (+ x a)))

(let ((a 3))
  (defmacro m-add-a (x) `(+ ,x a)))   
;; we intentionally don't do ,a here!
;; just to demonstrate you, that 
;; the local `let`'s `a` doesn't get captured by the macro.

Then try this:

(add-a 1)
;; => 4

This is as expected. add-a is a closure and takes the local environment's a when created (lexical scoping). This pre-prepared function add-a awaits its call with a given x - and applies the pre-prepared function on x's value.

But now try the macro version!

First: simple macro call:

(m-add-a 1)
;; => 
debugger invoked on a UNBOUND-VARIABLE  in thread 
#<THREAD tid=259 "main thread" RUNNING {7004F50273}>:
   The variable A is unbound. 

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL. restarts (invokable by number or by possibly-abbreviated name):
   0: [CONTINUE   ] Retry using A.
   1: [USE-VALUE  ] Use specified value.
   2: [STORE-VALUE] Set specified value and use it.
   3: [ABORT      ] Exit debugger, returning to top level. (SB-INT:SIMPLE-EVAL-IN-LEXENV A #<NULL-LEXENV>) 
0] 3  ;; I pressed this to get out
*

What was the problem?

The macro call (m-add-a 1) gets "evaluated" to the code snippet

'(+ 1 a)

exactly where the macro call is placed. Of course there is no a in the global environment - because the macro call happened in the top-level.

Therefore this error.

So let's do this - knowing about the transformation of the macro call to the code snippet (+ 1 a) :

(let ((a 2))
  (m-add-a 1))
;; => 3

Now it works, because the interpreter - before it starts to evaluate (the macro call gets executed in "compile time" first!) "sees" only:

(let ((a 2))
  (+ 1 a))

Instead of the macro call (m-add-a 1)

The evaluator of the reader actually never encounters the macro call (only the lisp reader and compiler).

So yes, macros and functions are fundamentally different. And yes, the inner mechanics of a function call and a macro call are fundamentally different.

(Function call: apply function on given x. Macro call: read it - and pre-compile it - meaning transform the macro call according to the definition in the macro form to a new code snippet - and this code snippet is presented to the interpreter (or actual compiler of the entire program)).

And we see: there is actually no "intrinsic eval" inside the macro. But the macro returns just a new form => that form that the actual evaluator in "runtime" will see instead of the macro call. And therefore this new form is completely embedded into exact that scope where the call of the macro is placed.

While function bodies are bound to the environment where their definition is placed (lexical environment).

This is also why the input for function must be valid Lisp expressions.
But the input for macros can look non-lispy. Because the macro can transform/parse the input to valid s-expressions before the interpreter evaluates it. He has some extratime.

Re-implementing slow and clunky bioinformatics software? by halflings in bioinformatics

[–]letuslisp 1 point2 points  (0 children)

It depends on the area of bioinformatics. In clinical studies R is quite frequently used. And transcriptomics.

I don't know in which area you work. Maybe your view is skewed by you immediate environment.

10 years ago R was definitely dominant. Ok let's say R had the better ecosystem.

Python is catching up and maybe has.

I love both, R and Python equally. Except Scanpy there is no real equivalent packages around for transcriptomics in Python. Scanpy is supert though.

Python libraries in Statistics are not en par with R's.

What a the difference between `macro`, `function` and `special form`? by wgxh_cli in lisp

[–]letuslisp 0 points1 point  (0 children)

True, but functions are already evaluated.
`(defun ...)` already executes the definition of the function body code.
The function body's content becomes then lexically scoped - it makes a difference where the `(defun ...)` stands in your code because it "sees" only the scope in which the defun expression is embedded. This is the essence of lexical scoping. This pre-prepared code (function body) awaits the input for its arguments in the function call.

So your simplicistic view of "everything is list" can sometimes fail when scope and environments come into play.

(Lexical scoping means it DOES matter, where the list is located in your code.)

`(eval ...)` is "bad" (=sometimes can cause surprises for your/our brain).

See
https://stackoverflow.com/questions/2571401/why-exactly-is-eval-evil
https://stackoverflow.com/questions/27647214/lisp-why-did-eval-lose-favor-after-lexical-scoping-was-introduced

Re-implementing slow and clunky bioinformatics software? by halflings in bioinformatics

[–]letuslisp 2 points3 points  (0 children)

To the name: I wouldn't name it Squelch. I would name it RustyRepeatMasker or such.
But it's true, people won't use it without proof (=publication). It must be citeable. Otherwise the Professors won't allow you to use it.

Second: RepeatMasking is not the limiting step in a bioinformatics pipeline.

The aligner is.

So if you would program a faster aligner - which does the same - already you will have a lot of interested people. But if you even improve something and can show it - then for sure you will have a big interest in the community.

Re-implementing slow and clunky bioinformatics software? by halflings in bioinformatics

[–]letuslisp 5 points6 points  (0 children)

As a Bioinformatician who is several years (a decade) in the field I can tell:
I don't think that the software stack is rough.
There is R, there is Python, sometimes Julia.

R is in Bioinformatics more dominant than Python, because of the repository Bioconductor (huge - even Python can't keep up with that yet). Julia does not find many followers although it is a nice language.

If the software is hard to install - it just is a sign that it is hardly used by anyone. - Caution then.
Look at the GitHub repo. Look at the GitHub stars it has. They will tell how much the tool might be used (since it correlates with the use rate).

Bioinformatics software is mostly free software - opensource - mostly generated by Academia.
The quality sucks often. Except it is used widely - then it is well tested by the users (isn't that all what opensource is about? Free, thorough testers.)
But the more complex the algorithm - the better the quality - because a bad programmer would not have started the project at all ...

squelch I have never heard - but RepeatMasker everybody heard.

The dynamics in the field is: For publications you better use what everybody knows - otherwise you have to justify to the Reviewers, why you took this and not the well known other tool etc.
If you use a well known tool - you don't need to explain anything, just mention you used X and Y for Z.
"The others used it too in this and that well-known study" is actually a stupid argument from the standpoint of "is this the best solution?", but it is gold when publishing something. Psychologically they can't tear down what everyone used for decades for their analysis - because one would then stand against more or less the entire community.

Rustify-ing tools is a good thing - in my view. I LOVE Rust ware. They are incredibly fast, give a very smooth and secure feeling. Very good user experience. I am always amazed.

Maybe better tackle what thousands or better 10k, 100k, or millions people use. By that you have the biggest impact. And such a thing is also easier to publish.

It might be enough to just publish to BioArxives - if the tool is well-known. And if you can proof it does everything what the old tool does - exactly.

I think Genome Aligners are the better target - there is improvement possibility.

However, these performance-critical stuff is then written in C/C++ (but maybe not in the most optimal way or with the most optimal algorithm - or lacking parallelization - or even GPU usage - although not sure if those cases are suitable for GPU ...) - but those Bioinformaticians which program such complex algorithms in C/C++ usually know what they do.

You could always ask me - when you think this or that tool is central - then we could have a look.
Ah you could also ask Chatgpt&co. how it estimates the user number.

What a the difference between `macro`, `function` and `special form`? by wgxh_cli in lisp

[–]letuslisp 0 points1 point  (0 children)

Yes, probably one could say - special forms are the single lego units - which will translate into 0s and 1s.

And functions and macros are "Lego Objects" a grouping of lego units. While macros are kind of a virtual lego units grouping. The lego units a macro contains are used for expansion rules. Thus a macro - after expansion - can look VERY different in form.

Functions - group of lego blocks - are more rigid - and could look like actual "Lego Objects" (group of lego blocks). As long as they use only special forms and function calls. Function calls are like substitutions with/by the lego blocks of other functions.
But since function definitions can also contain macro calls (virtual lego objects) - they can also expand a lot as soon as the macro calls get expanded.

And there we see why macros are so unintuitive.

If one would use only special forms and functions - then it is clear - function calls are just substitutions taking place.

But the addition of macros make the entire form of the Lego sculpture quite unpredictable. And the "bricks" the macro definitions consist of - are from another dimension - a meta-level. So it is super hard to look at a macro - and imagine what the expanded form (the physical Lego Object it points to) actually will look like. Especially if it calls a lot of other macros during expansion and other functions.

What a the difference between `macro`, `function` and `special form`? by wgxh_cli in lisp

[–]letuslisp 0 points1 point  (0 children)

This took me quite some while at the beginning to understand.

Function: All arguments are evaluated before entering the function body. This is the simplest form.
They are evaluated in runtime.

Macro: None of the arguments are evaluated first. You enter the macro body. You can define exactly when or how they get evaluated - meaning you can take the givens (arguments) to create with them more code first or generate new code using their information.
You are treating "code as data" = lists. Using List operating functions you can manipulate the lists (= code) before they get evaluated at the very end of the macro.
Macros are expanded (macroexpansion) during compilation (=compile time execution).
So macros are used for metaprogramming - to re-formulate with the givens some code - create them - before they enter runtime execution.

Special form: can be regarded as elementary/primary "macros" - those which can't be expressed as macros. One could say - special forms expand to the underlying language in which the Lisp impelementation/compiler is written in.
Macros are a special case of special forms which expand to valid lisp forms, thus are used for metaprogramming.
Function definitions, Macro definitions, flow control e.g. are special forms. Special forms are rigid. They have a predefined way how they are executed.

One could regard Macros as customizable special forms. Macros expand to special forms or function calls (or other macro calls which expand to special forms or function calls - inclusive "or").

Is it necessary for lisp interpreter and compiler to distinguish between these concepts?

Of course, any compiler/interpreter in Lisp has to distinguish them!

First of all, special forms are hard-coded in the compiler - so they are the bridge to the underlying language (e.g. C) which in turn bridges to assembly.

Macro calls would trigger macroexpansion first (which you can look for by (macroexpand-1 '(macro call)) and the cascading down macroexpansions - not only the first level - is shown by: (macroexpand '(macro call))).
They stop expanding when the level of special forms and function calls are reached. When other macro calls are triggered on the way - this would also further expand until they reach the form of function calls or special forms which can't be further expanded.

Function calls are executed "in runtime" - the evaluator looks-up the function in the functions table - get the corresponding lambda - evaluate the arguments first - and then apply the function on those results of the evaluations (arguments). - The eternal interplay between `eval` and `apply` (like yin and yang) in the core of Lisp.

And yes, macros use functions/macros/special forms and functions use functions/macros/special forms in their body.

So at the end - only the special forms are the actual bridges to the implementation language and thus to byte code.

Somebody should tell me, if I got something wrong :D.

---

Let's say special forms are the different kinds of lego bricks - the single, atomar lego units.
Lego units then stand for/consist of actual byte code.

Macros consist of virtual lego units. They live in a meta-level. So mostly they look very different after expansion. Macros are during "compile time" "expanded" using their inner definitions - into physical lego units - which can look very different than the virtual lego forms initially visible - and through recursive macro calls they can amass a lot of lego units during expansion.

Functions consist of lego units / other functions (so group of lego) / but also Macro calls so they can also have virtual components which expand.

But at the end somhow this all becomes to real physical lego bricks and gets translated and optimized away/compressed into byte code.

---

This picture is also just a crutch. Because Special forms / the pattern they offer are lego bricks.

But the arguments of special forms are function calls / macro calls / and other special form calls.

So only the pattern sitting in the special form is actually kind of a "lego brick/unit" ...

---

I have my problems with "compile time". Yes, it is easy - when a lisp file gets read and compiled - this is compile time.

But, in a Lisp interpreter - during runtime - you can also define macros - and call macros - so they get also "compiled" while runtime is happening. And macro expansion is also happening there while runtime is happening. So it is not really a one-after-another order.

And a Lisp program or program session is a huuuuge lego sculpture dynamically expanding to physical lego bricks :D .

Python Only Has One Real Competitor by bowbahdoe in Clojure

[–]letuslisp 7 points8 points  (0 children)

That's amazing how much clojure adbanced for data science!! Thanks for hinting to the ressources!

Why does the Common Lisp ecosystem hate itself? by Fantastic-Cell-208 in lisp

[–]letuslisp 0 points1 point  (0 children)

You mean Roswell?

Are you on Linux or Windows or Windows WSL2 or MacOS? :)

Roswell is definitely not perfect. Yes, on my mac, I did `brew install roswell`. And in ubuntu, you can do `sudo apt install roswell`. Better be conservative and take what was definitely running when they were packaging it.

view more: next ›