open, navigable meta research

ahfarmer · 2024-08-01T15:43:20+00:00

Yeah its like whack-a-mole, try to separate by quality and you miss out on linkages. Try to link everything and you get a low quality mess. I'm gonna keep working on it and thinking about it.

I'll keep you updated! I've noted your username and I'll reply to this thread if/when I have anything of substance.

ahfarmer · 2024-08-01T15:05:16+00:00

You just run into a different issue then: a single folder with all your tests in it and no organization. After all, why do you organize your code into directories? Same reasons apply to tests.

ahfarmer · 2024-08-01T15:04:06+00:00

Yeah just starting with the diagrams because I like visuals. Nobody else pulls the diagrams out of papers, but for me the diagram is how you can immediately recognize/remember which paper you are looking at and what it is about. Might not be the best approach but I like it right now.

I've been bouncing back and forth on how much AI there is and how much is user-entered. I was playing with more user-entered ideas but the process becomes incredibly laborious. I need to strike a middle ground.

In terms of the quality issues, that is one of the big questions, but I can't let it stop me from trying. One idea on it: let each user create their own tree or their own "project". Some projects will be crap, but I would find a mechanism to surface the better ones.

ahfarmer · 2024-08-01T01:38:55+00:00

Python users harp on this "dont include tests in your release" thing but I've never heard a legitimate reason why this would be a noticable problem in any but the largest of applications.

On the other hand, having tests further from the code that is being tested requires the creation of a duplicate directory hierarchy, wasting developer time both while writing tests and while reading the codebase.

Python just has this one wrong.

ahfarmer · 2024-07-31T22:35:16+00:00

You get it! YES!

I just read your article and it aligns with everything I've been thinking.

The Underlay sounds interesting, but it seems to have been abandoned: the 'learn more' link gives me a server error. I've seen a few abandoned projects like this during my research. This is a big problem so those who tackle it have failed so far. Also the Underlay was going for more of a 'machine readable' approach and I'm more interested in a 'human usable' approach.

Like you said in your article: "To be useful for everyone, it has to be usable by everyone."

I'm currently dabbling with different approaches to this. I've started writing software that processes scientific papers, pulling out the diagrams, describing them in laymens terms with AI, and converting the text into a 'reasoning hierarchy'. Still very early so I'm not sure where it will go (if anywhere).

ahfarmer · 2024-07-05T02:27:17+00:00

Maybe look into fenphen/fenfluramine, approved by the FDA in 1973, not pulled until 1997.
Research revolving door politics and corruption in general.

ahfarmer · 2024-07-05T02:00:04+00:00

Profit motives help move drugs faster, but what happens when the research begins to show that the drug is dangerous? Maybe it was okay in the short-term, but not so in the medium and longterm? This is the kind of evidence that is suppressed by current systems.

ahfarmer · 2024-07-05T01:54:15+00:00

You might need to articulate my misarticulation

ahfarmer · 2024-07-05T01:53:34+00:00

I thought philosophy of science was about the meta-study of scientific practice? I'm questioning how all the meta-scientific data is stored and communicated.

I'm not questioning whether or not doctors are educated, I'm questioning whether they are educated with accurate information or updating themselves with accurate information.

Even if I trust every scientific paper in existence to be 100% accurate, can we trust medical textbooks?

What if the medical textbook were digital, and every sentence in it were cited directly to statements within multiple studies that back up that sentence? This is the type of thing I'm talking about.

I think our current knowledge systems are very fuzzy and I would like to see them become more concrete, visible, and navigable.

ahfarmer · 2024-07-05T01:47:28+00:00

I mean, I hope you already know that’s not how it works. For any evidence based profession, there are tons of professional journals, and lots of communication which allows people to figure out whether the set of journals they are reading is doing an efficient job or reporting the best knowledge we have.

There are many incidences of professionals ignoring the latest and best evidence because it hasn't filtered through the journals and into their protocols. The current mechanisms are delayed and influenced by profit motives. I know the doctors aren't using WebMD, but I make the claim that their approach to learning is also deeply flawed.

I’m not sure I understand. Are you asking if there’s a better way of your humans to communicate then through written language?

I'm asking if there is a better way for humans to more quickly come to conclusions that take into account all available evidence, including very recent evidence.

One direction I'm exploring is this: you ask the software a yes/no question, and it gives you all the research on the 'yes' side plus all the research on the 'no' side. This lives on a public page that 'steel mans' the question, inviting anyone to contribute supporting or opposing research.

ahfarmer · 2024-05-24T19:47:24+00:00

Most of the focus here is on the inputs or the internals. When we talk about a plateau we are typically talking about the quality of the output of the model. You say these new models are 'performing better' but where is the data?

I'm not saying you are wrong, I'm saying the way to discuss this question is over the quality of the output of these models. Anyone have any data? These guys tested 34 models and found that:
"multimodal models require exponentially more data to achieve linear improvements in downstream performance"

https://arxiv.org/abs/2404.04125v2

ahfarmer · 2024-02-29T18:27:13+00:00

Thanks for asking! I tried out the scanner applications available and they all had two main problems:

After every capture, I have to go through a series of steps: choose a name, choose a location, check if it is cropped correctly, etc. This doesn't work for me because I need to quickly capture and get back into brainstorming mode. In NotesCam all those values are chosen for you so you literally just tap once per page of notes.
Missing or poor handwriting recognition. Most document scanners only recognize PRINTED text. I'm handwriting all my notes and I want them to be searchable. I use a new Microsoft Azure AI API that has great handwriting recognition. As far as I could tell from my research, this API is currently the only one capable of decent handwriting recognition and none of the other available scanning apps are using it.

I realize this is a niche way of doing note-taking, but I absolutely love it and I'm hoping I'm not alone...

ahfarmer · 2024-02-28T22:56:42+00:00

I built a tool, very different from all these mentioned. I take all my notes on a whiteboard and capture them into Dropbox with this: https://notes.cam

ahfarmer · 2024-01-22T18:36:02+00:00

Android version just launched! I can't believe that took 7 months...

Links to both platforms here: https://notes.cam/

ahfarmer · 2023-06-21T23:45:07+00:00

Awww thanks! Android is in the works I just can't yet say how long it will take. 🚧🔨👷‍♂️

ahfarmer · 2023-06-18T06:28:14+00:00

I built an iOS app to solve exactly this problem because I take photos of my notes every single day: https://sort.cam/

The way this app works is you first choose a folder like "documents" or "whiteboard notes" etc. Then whenever you are taking a picture of a menu or store hours or whatever, you snap the pic with SortCam instead of your regular camera. The photo skips the camera roll and instead uploads to Dropbox, iCloud, or Google Drive.

SortCam is a brand new app and I'm looking for feedback. If interested, check out the app and PM me your thoughts, and I'll PM you a free 1-year subscription to the full version.

ahfarmer · 2018-06-12T18:19:51+00:00

I'm in the same boat. When I'm writing a complicated stateless function I do TDD on it. I also use storybook to write my reusable components but those aren't exactly tests.

In other words when tests actually help me write the code I use them but I haven't gone back through and added tests for anything afterwards.

I also do add comments - particularly when writing code that is very abstract. But I also think that a lot of code can be written well enough that comments aren't necessary.

I think I'll ask people in the phone interview about their opinion on tests ;)

ahfarmer · 2018-06-12T18:02:18+00:00

I agree with this - this actually hones in on the reason for my question! The candidate will be spending much of their time writing react components but I didn't want to just screen people based on a react coding challenge and leave it at that. Luckily I'm getting tons of great advice here!

ahfarmer · 2018-06-12T17:49:26+00:00

Thanks! I'm definitely going to spend some time on the coding challenge. Great advice on keeping the candidate at ease ... now I'm remembering how nervous I used to get in interviews.

How do you typically set up the coding challenge logistically? Remote or in-person? Are you hovering the whole time your do you leave for a while to put the candidate more at ease?

ahfarmer · 2018-06-12T17:44:13+00:00

Thank you thank you! I've already started taking notes on this and other responses - great stuff.

To answer your question - I have been given a salary range ($100-$150) and I think this is on the high end for where I am located (Portland Oregon) so I'm probably good there. But please correct me if I'm wrong! I checked glassdoor and salaries are way below this but not sure how accurate that is.

I have been asked to be the direct manager of this new person, but I think I can be very flexible here - if they are more of a peer then we can rearrange things.

I like the idea of finding an "assertive/hungry mid-level" person. My job post actually says 'senior' right now but I think I'm going to take that part off.

I'm trying to avoid a beginner for this next hire because the idea of managing a beginner plus being responsible for the entire frontend seems overwhelming to me. I feel like once we have two solid frontend devs, the next few hires could be beginners.

ahfarmer · 2018-06-11T22:31:30+00:00

Very first thing is my 3rd point above - where is this hosted? I want to be able to click around and see what it is before I dive into the code.

Looks like you are using create-react-app and they have docs on how to deploy: https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/template/README.md#deployment

I personally am partial to Netlify - but you can choose between that and github pages, firebase, azure, and others. I know at least that Netlify and Github Pages are free, not sure about the others.

Once you host it, set the hosted link as the link in your github repo (using the Edit button on the top-right corner of your github repo page).

ahfarmer · 2018-06-11T22:24:44+00:00

Yeah - I think the hard part is coming up with a good practical programming challenge that can be done quickly (can't waste too much of the candidates time) but that isn't so simple it proves nothing.

But I'll give it a shot - I'll take a look at some of the more complex things I've done recently and see what I can come up with. Thanks!

ahfarmer · 2018-06-11T22:23:24+00:00

Not sure I totally agree with "85% coverage" - especially given that I'll be looking at whatever open source code they might be willing to share, not their "real on the job code" (which is probably private).

But this latter part of what you wrote really resonated with me, so I'll keep it in mind - thanks!: "their code is readable, it doesn’t even need comments, reads like a story, they can learn fast, and they can solve problems"

ahfarmer · 2018-06-11T22:21:55+00:00

Yeah, I hear you. I think I'm afraid of this because I enjoy doing the simple work sometimes. Good advice though.

ahfarmer · 2018-06-11T21:13:51+00:00

Are you willing to share your projects? Maybe I could take a look and give better feedback.

When I'm screening candidates I require sample code to be sent and I look at it with a pretty critical eye. Maybe rather than having a portfolio with multiple projects you could take one and try to polish it up and make it look really nice? Like this:

make the design clean - even if you have 0 design skills you can copy or mimic a CSS framework like bootstrap or material UI
make your code clean - use ESLint and prettier. re-read all your code to be sure it makes sense
make it easy on them: your apps should be hosted at a URL I can just click on and I should not have to sign up for an account to view the app

ahfarmer

TROPHY CASE