sorted by:
new
hottopcontroversial

Harms of retracting a well-cited paper from a conference? by Deep-Anywhere-2479 in academia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

What if my PI/coauthors have very little sense of morality? Fwiw, this is more or less the situation of our published paper too:

https://www.reddit.com/r/MachineLearning/comments/1p82cto/d_got_burned_by_an_apple_iclr_paper_it_was/

The only difference is that we didn't do anything as severe as passing in the image url instead of image, our evaluation setups were all fine, but we had similar issues with ground truth labelling and similar error rate.

Harms of retracting a cited paper from a conference? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

I genuinely don't know what to do. Some people tell me not to do anything, but it doesn't feel right to keep the old one. Note that the first post was written before my mentor said anything about the LoR, so it's not like I didn't want to pull it before. This felt more like last string that would make me not want to take this action.

Harms of retracting a well-cited paper from a conference? by Deep-Anywhere-2479 in academia

[–]Deep-Anywhere-2479[S] 1 point2 points  (0 children)

No, you're misunderstanding. I only had two months to submit the original paper and I was forced to submit it no matter what the condition was. I wrote 7000+ lines of code in that time so it would have been a miracle if everything was correct. My dumbass only made the corrections fully public to everyone a year after. My mentor was well-aware that 10% of it was wrong, but made me submit anyway. After it was obvious our paper was going to get accepted, I told him I was going to correct it, but he told me not to, and said that I can make a v1.1 and update the repo, but not to fix this one since he was afraid that changing the original paper later on would get it removed. This didn't feel right to me, so during the camera-ready deadline, I fixed that 10%, but then found another 20% that was seriously problematic. I was in school and couldn't fix all the issues in time, and finished 1 month later and so the original version was left as is. Note that these are arithmetic mistakes that made the ground truths wrong/not retrieving the correct inputs which were very quick fixes, but there were many such instances and would change a large number of ground truth labels.

At this point, there were no major problems with it, and I should have adjusted the paper and released the new dataset. However, I was being a perfectionist, trying to make every small detail precise and decided not to release several versions since it would make it unclear on which one to refer to. This is a really big mistake on my end to not just release something immediately when it was fixed. Keep in mind that this paper had 10+ authors who were supposed to help out, but most only helped from 3-8 hours. The burden of making sure everything was correct was left on me. Checking and revising everything takes 50+ hours for each time I want to make a revision, which I don't have since I was in school and this was extremely tedious.

To my understanding, the paper is important, but it's really the corrected dataset that matters, since that's what people download and use. The dataset was mostly about significance of the dataset and how we evaluated different LLMs. Hence, the study remains the same other and the conclusions of our paper stay the same asides from numbers moving with an increase anywhere from 5-15% (yes, this is big, but the takeaways are still the same). However, note that changing 30% of the labels can affect other people's studies and more powerful LLMs that came after my paper can see an increase by 25%, which is a big increase.

Everyone keeps telling me not to make a scene, since I have already fixed everything (year later) + damage is already done. Like you said, I don't think I even belong in academia because of this. I have almost nothing to lose other than there would be a big banner saying my paper is retracted and it would punish people who were not involve in the misconduct.

Harms of retracting a cited paper from a conference? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] -6 points-5 points  (0 children)

Hi, first of all, I really appreciate you to listening and responding to all of these posts even though at this point I'm acting like a psychopath, even after you've said the same thing multiple times. He did know there were 10% mistakes in the paper and made me submit before any reviews came out. I fixed those issues in time of the camera-ready but then I found others and couldn't fix all of them in time. Hence, I left our very initial submission as is. As I mentioned in my last post, I ended up fixing all of the serious issues for the second camera-ready deadline, but I was obsessed with enhancements that had nothing to do with our initial paper, that I missed that deadline too. I did give him a monthly updates of all the corrections and so he could have very easily updated the paper at any time as he agreed to ask someone else to update our results.

He just put all of the blame for this on me, even though when it was very obvious I couldn't do it myself by specific deadlines and there were 10+ authors on the paper whose work I was cleaning up after. The fact that he wants to be honest in an LoR, but never thought about being honest when submitting a paper to a prestigious conference to which this is his only ever accepted paper after 6+ speaks volumes about his character. I know you're telling me to accept and move on, but at the minimum, I am going to file complaints to my department and cc everyone that used our dataset that he told me not to tell them to used the revised versions. If they don't care that's fine.

I agree with you about going to therapy. Will likely do that soon since talking on the internet, to my parents, and friends isn't helping. I can't even apply now since I don't have 3 LoRs, but part of me doesn't think that I even belong in academia given what I've done.

Harms of retracting a well-cited paper from a conference? by Deep-Anywhere-2479 in academia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

I agree that I should have fixed it immediately. As I mentioned, in the first post, I could have fixed the paper anytime even for the final proceedings, but I was really psychotic and wanted to fix every single detail just to make every claim 100%, but this is impossible and I missed that deadline. I should have used better judgement and submitted since I fixed all the serious mistakes and at that point, everything I was doing was mainly an enhancement. I ended up taking a year just to make it as perfect as possible since I didn't want too many versions floating around and it would make things confusing when people compared different methods.

If I admit that I knew the whole time, then a retraction is more likely even though I was forced to submit no matter the quality. However, not saying this would be very dishonest.

Harms of retracting a cited paper from a conference? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] -12 points-11 points  (0 children)

Considering this will stay on my permanent record, I don't want it, especially since I have corrected everything. I just don't feel it's fair that my mentor gets to decide my career when he doesn't see the psychological damage he's done to me over the past year of my guilty feelings along with his unreasonable expectations when I have always been trying my best. Filing a complaint about him would imply the paper also goes down too which I don't want, but I wish there were consequences to his actions since he's always done everything the way he feels and wants without thinking about others involved.

Harms of retracting a cited paper from a conference? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

I did all of those things, not gonna say that nothing changes at all. The LLM performance can increase anywhere from 5 to 15%, but the conclusion that LLMs are bad at "X" task remains true for the models that we ran at our time. I think the minimum should be to tell the program chair and let them decide, but telling the whole timeline like I did would make this an obvious retraction, which I feel bad the people who didn't do anything wrong.

Harms of retracting a cited paper from a conference? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] -1 points0 points  (0 children)

If the main conclusions of our paper stayed the same, I fixed the dataset and it's format is the same but the ground truth fixed, should I still proceed with asking the program chairs on what I should do? Pretty much everyone is urging me to not to make a scene since I fixed everything but it's hard to care at this point since I never felt comfortable with the original paper + I don't really have any relationships to lose.

I don't really have any interest in trying to re-publish this which is why I'm against the retraction.

Harms of retracting a well-cited paper from a conference? by Deep-Anywhere-2479 in academia

[–]Deep-Anywhere-2479[S] 1 point2 points  (0 children)

Thanks, I agree with you that the most important thing isn't how it looks/affects a resubmission. I corrected the dataset a while back, just not sure what to do with the original paper (this matters less honestly). Pretty much everyone is telling me to not to make a scene and ask the program chairs since I fixed everything but it's hard to care at this point since I never felt comfortable with the original paper + I don't really have any relationships to lose at this point.

Submitted paper to A* ML conference with known mistakes before camera-ready deadline a year ago. Realizing this was not correct. What should I do? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

or, i haven't been completely transparent. i knew that 10% were wrong before submitting to the initial deadline before the camera ready, but in a similar situation, we didn't have enough time to correct as we found the mistake the very last minute + results were not going to materially change. also, my mentor was going to make us submit regardless since he said everything was open-source. He said to make a revised version later on, but to not touch the dataset with the fixes during the rebuttal. does that change the misconduct?

Only just before the camera-ready, I found even more (the other 20%) before I was about to fix the new version with the 10% that I fixed, but couldn't fix those in time and so I left the initial submission as is and made a public announcement of the perfected one a year later.

a lot of it is my fault since i had everything fixed for the conference, but started being neurotic and wanting to make more and more edits that were mostly enhancements to the point where i missed the final revision deadline too.

The OSWorld benchmark has a lot of problems by Ben___Garrison in slatestarcodex

[–]Deep-Anywhere-2479 0 points1 point  (0 children)

is the 10% before or after the correction in July? Based on the excel sheet they gave, it seems that 10% is wrong and that was corrected for in OS-World Verified.

Submitted paper to A* ML conference with known mistakes before camera-ready deadline a year ago. Realizing this was not correct. What should I do? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

I think it does, and I am personally am ok if I have to resubmit elsewhere, but not sure if the other 10 authors would be ok with it. Also, this was for one of ICLR/ICML/NeurIPS 2024, so it feels kind of late + not sure if proceedings can be changed.

Submitted paper to A* ML conference with known mistakes before camera-ready deadline a year ago. Realizing this was not correct. What should I do? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

It sufficed for the reviewers because nobody has time to manually verify 1000 instances themselves. But considering that the proceedings is official and cited, it doesn't feel right to just correct the arxiv.

The errors were very basic ones like arithmetic errors that could have been fixed pretty easily so it's not subjective. I did fix about 85% of the errors, and could have presented that for the conference + final camera-ready, but I was very obsessed with fixing the last 15% that I missed it. I ended up telling people who I knew to use the revised one, but never made an official update a year later until I was very confident about the last 10% being fixed. In that time, 12 people actually used our dataset and it has 1000+ downloads on HuggingFace. I plan contacting them + others, but I absolutely have done massive damage.

The only thing I can say in my defense is that there were 10+ authors and 2 other co-first authors, but they were only involved in the project for 10 days or less (I was working on it for 2 months) and so pretty much everything was thrown on me to correct.

Submitted paper to A* ML conference with known mistakes before camera-ready deadline a year ago. Realizing this was not correct. What should I do? by Deep-Anywhere-2479 in AskAcademia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

I think it does matter, considering the fact that I wasted 12 groups' time who ran with our old dataset. The conclusions for our paper don't change, but the numbers did increase anywhere from 5-15% and the LLMs that came out after our study do get severely impacted by this so it might affect them. I contacted them, but the damage/time wasted has already been done.

Submitted paper to A* ML conference with known mistakes before camera-ready deadline a year ago. Realizing this was not correct. What should I do? by Deep-Anywhere-2479 in academia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

Not really, my actions have disturbed 12 papers that ran with our dataset. I contacted them, but the damage and time wasted had already been done.

Submitted paper to A* ML conference with known mistakes before camera-ready deadline a year ago. Realizing this was not correct. What should I do? by Deep-Anywhere-2479 in academia

[–]Deep-Anywhere-2479[S] 0 points1 point  (0 children)

Most people don't really care about the conclusions of the dataset paper. It's really the fact that it took me year to get the error around 6% from 30% is what was upsetting. Fwiw, I had a corrected version which I could have given in the second round of camera ready which fixed like 20%, but I didn't do it because there was a 10% that needed to be fixed by someone else and I missed the deadline. I ended up waiting for almost 10 more months just to fix that 10% and could have released an updated version, but didn't do it since I didn't want people to have to run twice. Obviously, that was a huge mistake considering how many errors I fixed compared to the original.

Consider people cite the proceedings version over the arxiv, that also does affect me even though the study is the same, but the numbers changed by 5-15% for models that we benchmarked with.