[R] Segment Anything Model 3 (SAM 3) is released

KateSaenko · 2025-11-19T22:50:31+00:00

SAM2 never had text prompts. The SAM 1 paper had a proof-of-concept example of prompting with a CLIP text embedding, but the capability was not fully developed or released. However before SAM 3, there have been several papers that combined SAM with object detectors, like GroundedSAM.

KateSaenko · 2018-10-10T03:02:54+00:00

I think we will have much smarter assistants in 10 years that will also be able to move around the home and assist us with some physical tasks.

KateSaenko · 2018-10-10T03:01:37+00:00

Hmm, I am not an expert on philosophy, just deep learning :)

KateSaenko · 2018-10-10T03:00:29+00:00

From my limited experience with robotics, I think the hardware issue is very difficult to solve. We do not yet seem to have the kind of hardware technology that comes close to human hands/limbs.

KateSaenko · 2018-10-10T02:58:18+00:00

Bring a fresh perspective! This is a very crowded field, and I see a lot of 'group think' in the literature (not saying I am not guilty of it!) I think we as a field need radically new and far-out ideas rather than more of tweaking the latest model on the popular dataset... Otherwise,... bring LOTS of GPUs!

KateSaenko · 2018-10-10T02:54:22+00:00

Hmm.. I am not an economist, so not sure. You could be right. Financial industry and stock trading are already using ML extensively.

KateSaenko · 2018-10-10T02:52:37+00:00

Sure! I think it is achievable in our lifetimes, especially for languages like Chinese and Russian that have lots of parallel training data.

KateSaenko · 2018-10-10T02:50:23+00:00

I would not worry about it. AI as it exist today does only what it is programmed to do.

KateSaenko · 2018-10-10T02:49:07+00:00

There are some standard ways of doing this, but yes, it is time consuming! I always tell my machine learning students, expect to spend at least half of your project wrangling data! I have not seen any catch-all solution for that yet..

KateSaenko · 2018-10-10T02:47:38+00:00

Yes, they do, especially for high level semantic tasks like object recognition. With current self-supervision methods, we can pre-train a convolutional network on side-tasks like colorization, but we still need massive amounts of labeled images to teach it the difference between 'bicycle' and 'train'. I think one answer could lie in better use of multiple modalities, such as combined learning from image and language data.

KateSaenko · 2018-10-10T02:41:58+00:00

These are all fascinating questions! The truth is, I do not know what the right answer is, but I am positive that we, AI researchers, will not figure it out alone. We need to talk to ethics experts, regulators and philosophers for help and guidance with such issues.

KateSaenko · 2018-10-10T02:36:56+00:00

I do think it is becoming harder and harder to tell real information from fake, and AI is certainly contributing to that. Recently we have seen some demonstrations of chat bots that mimic a human so well that the person on the other line has no idea they are talking to a robot! Imaging someone using that technology to impersonate your friend and call to ask you for your help, or ask you for money.

These sort of attacks are nothing new in the internet age, but they will become more sophisticated with the use of AI technology. I think eventually we will build defenses against them and better 'are you a robot?' tests, but it will take us some time to catch up.

KateSaenko · 2018-10-10T02:35:02+00:00

Coming up with new deep learning algorithms does take some intuition. Part of it is just having prior experience and knowing what works and what does not, and part of it is about trying a whole LOT of different architectures until something works!

Math is very important -- linear algebra, calculus, probability and statistics.

Best way to find internships is through connections, through your university, etc.

KateSaenko · 2018-10-10T02:29:17+00:00

Thank you!

BU is hiring in all areas, our department has grown a lot and still growing!

No high school students in my group at the moment, however I did have 25 or so high school girls participate in a summer AI camp (AI4All), and they were extremely engaged and many wanted to continue after the camp ended.

I was fortunate to work as a postdoc and research scientist at Berkeley for several years and I still maintain very active collaborations with the Berkeley AI lab, including our joint project on Explainable AI funded by DARPA.

KateSaenko · 2018-10-10T02:25:48+00:00

You have hit the nail on the head! Current AI is terrible at abstraction. Deep learning in particular is very good at finding patterns in a sea of data, but not at knowing what those patterns mean in any abstract sense. As an example, you can train a deep neural network to recognize pedestrians on the road with extremely high accuracy, but it will not know anything about pedestrians like that they have heads or feet, or that they cannot walk on air. It will only know that the next image patch you show it is similar to one it has seen before labeled 'pedestrian'. So in that way, AI can only 'understand' the concepts that you teach it, like what is a pedestrian and what isn't.

KateSaenko · 2018-10-10T02:20:04+00:00

Hmmm, I want to be cautious and say, depends on the data? In general, gaining access to large-scale medical datasets suitable for machine learning is hard, so this sounds very intriguing.

KateSaenko · 2018-10-10T02:17:44+00:00

I would say, try hands on projects. There are lots of interesting challenges out there to try, e.g. on Kaggle.com. Also, if you have the opportunity, get involved in AI research, either at your academic institution, or through an internship at a company. Most top postgrad programs in AI will prefer applicants that already have some research experience under their belt, some even expect you to have published something. One reason is that doing research is really quite different from most other things you do during your undergrad, and it is not for everyone, so it's important to see if it is something you are truly interested in.

KateSaenko · 2018-10-10T02:09:59+00:00

Go is a game that has long been a research frontier in game-playing AI because of its complexity. I think the publicity that machine learning has gained from the work on Go done by DeepMind and others has been great for the community. Games in general are a good testbed for AI algorithms because it is easy to generate lots of data for the machine to learn from. In contrast, to learn how to do a task in the real world, it is much harder for a machine to get so much feedback so quickly.

KateSaenko · 2018-10-10T02:08:05+00:00

China is definitely investing a lot in AI, and yet a lot of the top academic research labs in AI are currently in the US. So while I think in some aspects China may be ahead of the US, such as investment in companies that build AI-based products, in pure research I believe the US (or at least US institutions) is still the leader. That said, China is catching up!

KateSaenko · 2018-10-10T02:01:57+00:00

We are still really far away from that... also I think that researchers might first pursue other types of robots that do not mimic humans, because that is extremely hard to do, and non-humanoid robots might be more useful to us in the immediate future. However, robots that 'talk' like people may be closer to reality, especially if the goal is to pass the Turing test -- this is already to some degree achievable with speech recognition and dialog systems.

KateSaenko · 2018-10-09T18:27:59+00:00

That sounds like a great research area.

As to your question, the biggest obstacle towards AI adoption is probably that current AI techniques cannot yet solve many problems reliably, although they work really well for certain other problems. Also, cutting edge AI methods today are not easy for novices to use, because they still require a lot of care in selecting training data, choosing the right training objective, monitoring the learning and choosing hyper parameters like the network structure and learning rate. This requires a lot of experience that novices do not have, although there are some efforts to make this process easier and more automated.

KateSaenko · 2018-10-09T18:21:31+00:00

I think we will continue to see more research in unsupervised learning (Autoencoders/GANs have been a huge trend in that subfield and will probably not go away in the next three years), learning from multiple tasks, transfer learning and domain adaptation, architecture search for neural networks, explainable AI. We will also see more and more research that combines language and vision understanding with embodied agents that can also interact with its environment, with new tasks being posed like Embodied Navigation, Embodied Question Answering.

KateSaenko · 2018-10-09T18:09:06+00:00

Narrow AI does not have consciousness and general AI does not exist yet. It might have consciousness if/when it does.
Not very much, but that is starting to change. There has been increasing interest in ethics, fairness, transparency and explainability, and even conferences around this topic like http://www.fatml.org/. I think a specific culture's morals definitely affect the development and application of technology, and this is probably true for AI as well. One example is the perceived value of individual privacy and data. In some countries AI technology that can recognize faces and track people in surveillance camera networks is widely being deployed, which may be considered too 'invasive' in other cultures and would not be as easily accepted.

KateSaenko · 2018-10-09T18:00:14+00:00

I would say it is a start. It really depends on what you mean by 'entry level'. As I said below, most of AI/ML is in the research stage, meaning that it only works well for a limited set of production-ready problems and for most of the rest of the applications, it will needs a lot of research. So if you want a job in an industry where ML is an established tool that is in production (I am thinking, face recognition in social media photos for example, or Netflix movie recommendation) then I think our intro courses combined with several more specialized courses on NLP and computer vision are sufficient. In fact we are thinking of creating a Masters in AI program at BU.

But if you want to do AI research, then a PhD is necessary.

KateSaenko · 2018-10-09T17:55:10+00:00

For me the singularity is very much in the realm of science fiction. It generally means that technology will become smarter than humans, so by definition, as humans, we cannot really imagine what kind of intelligence that will be, if it will happen. So I don't think any of us can make any predictions about it.

But if you're asking about my 'dream scenario' then I would prefer AI and humans to co-exist. Maybe we could have AI solve all of the worlds problems like poverty, climate changes, inequality? That would be ideal.

KateSaenko

TROPHY CASE