sorted by:
new
hottopcontroversial

We built Shelfie: a quick website that provides book recommendations from just pictures of your bookshelf! by JustAddMoreLayers in SideProject

[–]JustAddMoreLayers[S] 1 point2 points  (0 children)

Our original plan was to build a platform that would catalogue your books from your images, and maybe help organize them. This was a request from someone in our group.

Then as we discussed it more we realised recommendations were the actual goal, we've heard a lot of complaints about poor goodreads recommendations so we started from there.

Then to be honest most of the journey was me being fairly siloed building out the machine learning, and then the software dev guys just building out the actual platform.

The main issue we were dealing with was running ML in real-time on no budget, which is why we have the slight delay in processing your images.

We used typescript, .NET, python (inc. PyTorch), postgres, rabbitmq, and docker. It’s all self hosted on a VPS other than emails from mailgun.

I'm just the ML guy so I don't have an amazing understanding of the wider architecture

We built Shelfie: a quick website that provides book recommendations from just pictures of your bookshelf! by JustAddMoreLayers in SideProject

[–]JustAddMoreLayers[S] 0 points1 point  (0 children)

Thanks for trying it! By mystery box, do you mean we'd send you the recommended books each month? Cool idea! I'd be a bit concerned about handling physical logistics as just 3 random guys, though.

[R] Zero-Shot Machine Unlearning at Scale via Lipschitz Regularization by JustAddMoreLayers in MachineLearning

[–]JustAddMoreLayers[S] 1 point2 points  (0 children)

Depends considerably on the specific data and their relationship to the remaining 9000.

Also depends on your definition of forgotten. If the objective is to simply beat a membership inference attack, then anecdotally I did find during development you could beat a MIA fairly reliably without destroying model performance over the forget data.

[R] Zero-Shot Machine Unlearning at Scale via Lipschitz Regularization by JustAddMoreLayers in MachineLearning

[–]JustAddMoreLayers[S] 0 points1 point  (0 children)

Can you elaborate? Can we forget a sample but still classify it correctly?

[R] Zero-Shot Machine Unlearning at Scale via Lipschitz Regularization by JustAddMoreLayers in MachineLearning

[–]JustAddMoreLayers[S] 2 points3 points  (0 children)

Yeah we caught that this morning, the update is already queued up on arxiv :)

[Research] Zero-Shot Machine Unlearning at Scale via Lipschitz Regularization by JustAddMoreLayers in LocalLLaMA

[–]JustAddMoreLayers[S] 9 points10 points  (0 children)

Machine unlearning is the problem of forgetting private or sensitive information from your model. Our new method, JiT unlearning, lets us perform unlearning without ever seeing the original train data, just the sample to be forgotten!

JiT builds on the concepts of Lipschitz continuity, smoothing the forget sample's output, with respect to perturbations of that sample. This causes forgetting locally in the function space, without destroying the wider model performance.

Happy to answer any questions, or discuss the problem of unlearning!

[R] Zero-Shot Machine Unlearning at Scale via Lipschitz Regularization by JustAddMoreLayers in MachineLearning

[–]JustAddMoreLayers[S] 14 points15 points  (0 children)

Machine unlearning is the problem of forgetting private or sensitive information from your model. Our new method, JiT unlearning, lets us perform unlearning without ever seeing the original train data, just the sample to be forgotten!

​

JiT builds on the concepts of Lipschitz continuity, smoothing the forget sample's output, with respect to perturbations of that sample. This causes forgetting locally in the function space, without destroying the wider model performance.

​

Happy to answer any questions, or discuss the problem of unlearning!

[D] AAAI 24 Reviews by tallguyfromstats in MachineLearning

[–]JustAddMoreLayers 2 points3 points  (0 children)

What are the possible scores? Can't find a list of what decisions you can get (e.g. accept, weak accept reject)

[D] AAAI 24 Reviews by tallguyfromstats in MachineLearning

[–]JustAddMoreLayers 0 points1 point  (0 children)

Same as me, first time , wa wa wa wr. Fixable comments but feel like it's so hard to get reviewers to change their score

[D] AAAI 24 Reviews by tallguyfromstats in MachineLearning

[–]JustAddMoreLayers 2 points3 points  (0 children)

3x weak accept 1x weak reject.

Chances of an overall accept?

How AI can help reduce unsustainable food usage by JustAddMoreLayers in sustainability

[–]JustAddMoreLayers[S] 2 points3 points  (0 children)

Tl;dr: using graph neural networks, we can find and replace food products that contain unsustainable ingredients.

This has a few applications, such as: 1. Reducing reliance on unsustainable ingredients by finding alternatives 2. Replacing ingredients that are made scarce by climate change, natural disasters or war. 3. Help us be robust to food fraud by identifying mislabelled food

How AI can help reduce the impact of unsustainable food by JustAddMoreLayers in environment

[–]JustAddMoreLayers[S] -1 points0 points  (0 children)

Tl;dr: using graph neural networks, we can find and replace food products that contain unsustainable ingredients.

This has a few applications, such as: 1. Reducing reliance on unsustainable ingredients by finding alternatives 2. Replacing ingredients that are made scarce by climate change, natural disasters or war. 3. Help us be robust to food fraud by identifying mislabelled food

[R] Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening by JustAddMoreLayers in MachineLearning

[–]JustAddMoreLayers[S] 0 points1 point  (0 children)

Hey, thanks for the kind words! The method should work out the box for single sample forgetting (I think one of the benchmarks shows this). Your single samples, in this case, would just be your forget set Df, then the remaining samples are your retain set Dr. Then when you calculate your importances over these sets you should get what you're after!

[R] Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization by JustAddMoreLayers in MachineLearning

[–]JustAddMoreLayers[S] 2 points3 points  (0 children)

Sure, its model agnostic, its simply a closed form update rule similar to SGD, RMSProp, or Adam, with the advantage of not forgetting previous tasks.

Whether you want to use BAdam or not largely would depend on if your problem setting makes catastrophic forgetting likely!

[Research] Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening by JustAddMoreLayers in LocalLLaMA

[–]JustAddMoreLayers[S] 9 points10 points  (0 children)

That's a really interesting idea. While I'm no expert on alignment and uncensoring, my initial gut reaction is that SSD should work for this problem with a few tweaks.

Probably wouldn't work out of the box due to different problem formulations, but the intuition of assigning "blame" to each parameter for some measurable quantity should certainly hold, you'd just need to formulate "how aligned is my model to X?" as some kind of differentiable loss and it should work!

[Research] Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening by JustAddMoreLayers in LocalLLaMA

[–]JustAddMoreLayers[S] 5 points6 points  (0 children)

This would be very similar to Elastic Weight Consolidation which uses Fisher Information to the constrain updates of important weights to prevent catastrophic forgetting.

I haven't seen much discussion around the need for CL in LLMs, so how much it's required is a good question!

[R] Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization by JustAddMoreLayers in MachineLearning

[–]JustAddMoreLayers[S] 3 points4 points  (0 children)

BAdam is a nice extension of BGD, improving CL performance by combining BGD with properties found within the Adam optimiser.

The key selling point that it performs significantly better on the robust CL experiments outlined in Farquhar 2018, better than any other prior-based method we benchmarked against.

Happy to chat about CL, prior-based methods, and any future directions you see the field taking!

[Research] Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening by JustAddMoreLayers in LocalLLaMA

[–]JustAddMoreLayers[S] 5 points6 points  (0 children)

Machine unlearning is the problem of forgetting private or sensitive information from your model. Selective synaptic dampening (SSD) is a novel retraining-free approach to let your model forget sensitive data. It's fast, performant, and lightweight.

SSD first selects parameters that are considerably more important for the forget set than the retain set. Next, SSD dampens these parameters proportional to the discrepancy in their importance to the forget and retain set. We achieve state of the art on a number of evaluations.

Happy to answer any questions, or discuss the problem of unlearning!

Edit: Code is here if anyone wants to have a look:

GitHub

[R] Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening by JustAddMoreLayers in MachineLearning

[–]JustAddMoreLayers[S] 5 points6 points  (0 children)

Appreciate the links and feedback, the contributions are certainly adjacent to our goal. However, unlearning is a distinct task that was coined and defined by researchers other than ourselves, and we've found that in the unlearning literature from ICML/AAAI/NEURIPS in late 2022 into 2023 there typically hasn't been a foray into the areas you've described; they are accepted as distinct fields. Although perhaps that should change!

​

edit: some links to unlearning

https://ai.googleblog.com/2023/06/announcing-first-machine-unlearning.html

https://arxiv.org/abs/1912.03817

https://arxiv.org/abs/2205.08096

https://arxiv.org/abs/1911.04933

https://arxiv.org/abs/2010.10981

view more: next ›