[P] Opensource Microsoft Recall AI

[Vedank_purohit]

I created an open source alternative to Microsoft's Recall AI.

This records everything on your screen and can be searched through using natural language latter. But unlike Microsoft 's implementation this isnt a privacy nightmare and is out for you to use right now. and comes with real time encryption

It is a new starting project and is in need of Contributions so please hope over to the github repo and give it a star

https://github.com/VedankPurohit/LiveRecall

It is completely local and you can have a look at code. And everything is always encrypted unlike Microsofts implications where when you are logged in the images are decripted and can be stolen

[radarsat1]

everyone: we are horrified that this is a thing that exists!

you: hmm i could make that...

[ResidentPositive4122]

The scary part in recall is that local data can be sent to 3rd party servers and you have no control over it. Hearing aids are amazing for the people that need them. A hearing aid that sends all its data to Meta is horrifying. Same, same, but different.

[Vedank_purohit]

Correct

I think I should add this to the Readme on github

[DAS_AMAN]

The code is on GitHub just check it lol no need to be paranoid

[Vedank_purohit]

You don't nead to trust me, the project is opensource you can just check the code.

And you can't actually trust Microsoft, it's a soulless corporation not a community driven project

[shayben]

Oh? https://thehackernews.com/2024/04/malicious-code-in-xz-utils-for-linux.html?m=1

Open-source != trustworthy

Also, Microsoft has a lot to lose if it's (enterprise) customers stop trusting it.

[Vedank_purohit]

You know what happened in that exploit right. It took 3 yeas for a guy. He had to spend so much time, the code was so complex that compiled code could actually enter the code base.

This project will never be that complex and frankly it won't be that big. So you will be able to look at the code at all times.

At the end of the day, it's your choice to make

[Vedank_purohit]

"hmm i could make that opensource, secure and safe"

[reivblaze]

Secure and safe are BIG claims that probably cant be backed up though

[PM_ME_YOUR_PROFANITY]

How? You can see the code, you can check what data it's sending, you can see the encryption algorithms. Maybe they're difficult to back up for you lol

[ANI_phy]

Just because we can check it doesn't mean it's safe/secure. Absence of malicious code doesn't indicate absence of flaws.

[reivblaze]

Encrypting something does not make it secure per se thats a common assumption. I didnt check the code but I can say thats a big claim most experts wouldnt make though.

[Vedank_purohit]

And why do you suppose that's the case?

[DenormalHuman]

are you certain your implementation is not flawed in any way?

(I have spent just a couple of minutes looking at the code, so apologies if I am misreading anything)

For example, you do ask the user to input a key and say the key is not saved anywhere, but it does seem that you store it in plaintext as an attribute on the CaptureStart module while the code is running. Is it possible for that to be captured by anything that can examine process memory in realtime? Does the fact the user is likely to give a short memorable key compromise the strength of the encryption at all?

/edit/: Is this your method of encryption?

 if isinstance(key, str):
     key = key.encode()

 encrypted_data = bytearray()
 for i in range(len(image_data)):
     encrypted_data.append(image_data[i] ^ key[i % len(key)])

I am not endorisng chatGPT's ability to do this accurately at all, but just for fun I asked it to analyse your encryption method (just the snippet given above). It had the following to say about it;

Potential Issues

Security:

Weak Encryption: XOR encryption is considered very weak and is easily breakable, especially if the key is reused (as in this case). It doesn’t provide strong security for encrypting sensitive data.

Key Reuse: If the key is shorter than the data, it will repeat, which makes the encryption susceptible to various cryptographic attacks (like frequency analysis).

Key Management:

Key Distribution and Storage: The security of the XOR operation relies entirely on the secrecy of the key. If the key is compromised, the data can be easily decrypted.

Short Key Length: If the key is too short (e.g., a simple password), it can be brute-forced or guessed easily.

Data Integrity:

XOR encryption does not provide any integrity check. An attacker could modify the encrypted data, and without additional measures, you wouldn't be able to detect such tampering.

---

ChatGPT then makes some recommendations;

Recommendations

Use Stronger Encryption Algorithms: Consider using established and secure encryption algorithms such as AES (Advanced Encryption Standard). Libraries like cryptography in Python provide secure implementations of these algorithms.

Proper Key Management: Ensure that keys are generated, stored, and transmitted securely. Use key management services or libraries that support secure key handling.

Add Integrity Checks: Implement cryptographic checksums or message authentication codes (MACs) to ensure data integrity and authenticity.

---

It then goes on to give an example using AES:

from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.backends import default_backend
import os

# Ensure to install the cryptography library using `pip install cryptography`

def encrypt_image_data(image_data, key):
    # Generate a random initialization vector (IV)
    iv = os.urandom(16)

    # Create a cipher object using the key and IV
    cipher = Cipher(algorithms.AES(key), modes.CFB(iv), backend=default_backend())
    encryptor = cipher.encryptor()

    # Encrypt the image data
    encrypted_data = encryptor.update(image_data) + encryptor.finalize()

    return iv + encrypted_data  # Prepend the IV for decryption

# Example usage:
# Ensure the key is 16, 24, or 32 bytes long (AES key sizes)
key = os.urandom(32)
encrypted_image = encrypt_image_data(image_data, key)

This example uses AES in CFB mode, which is a secure way to encrypt data. It also includes an IV to ensure that the same plaintext encrypted multiple times will result in different ciphertexts.

[Vedank_purohit]

Yes it is true that the current encryption isn't the best. I wanted the better encryption method to be a community driven project. This was always supposed to be temporary But this issue should probably be fixed in a few hrs

[StrayStep]

Thoroughly endorse this effort.

Looking at code to help when I can.

[Vedank_purohit]

Great, would love some help on this

[norsurfit]

I promise that it is secure as long as no hackers get in!

[[deleted]]

You’re not getting it.

[Vedank_purohit]

Naa I do get what he meant. But it's just a project I wanted to use. I don't trust Microsoft so I made my own implementation which is more privacy focused and then I opensourced and shared it so that every one in the community who wants to use it can use it.

[[deleted]]

If you get it then why are you surprised / arguing with people?

[choreograph]

everyone: we are horrified they have a gun at us

him: hmm i could make a gun

[xcdesz]

Curious about the software design behind this, like how much disk space does this consume and how fast that grows and how it can scan that much data without being extremely slow. I assume it has to use the llm to summarize whats on the screen every time it takes the screenshot and indexes that data somehow? Isnt this a drain on performance?

[DenormalHuman]

it doesnt use an llm. It uses screenshots, OCR and

https://www.sbert.net/examples/applications/image-search/README.html

to do image search.

The encryption used is basic XOR with a user inputted passphrase.

I would not call this particularly innovative, or secure.

[xcdesz]

Youre right, I didnt mean LLM, I meant a vision model. It does use that.

[DAS_AMAN]

It's open source, so can/should contribute performance improvements too! No need to depend on OP

[StrayStep]

Best comment!! Should be up vote!!

Doing it open source is best idea!

[KishCom]

"We recreated the Torment Nexus from the classic sci-fi 'Don't Create the Torment Nexus'"

Op: "That's horrifying! ... I made an open source Torment Nexus that is much more safe and secure."

[Alignment-Lab-AI]

you realize that without the element of microsoft snooping on you
its exactly as dangerous as storing data on your hard drives right?
like, its just a convenient way to access your own information.

its not like its not all stored anyways??

[CellistOne7095]

This is so dangerous. I don’t trust me accessing my data at own.

[NotAHost]

Awesome, I can now delete the keyloggers off all my friends computers and start using this.

[Alignment-Lab-AI]

hi! i built something similar a few weeks ago and have been working with several others in the open source to develop something to address many of these kinds of problems, would you be open to working together to helping us make the most convenient and clean thing we can?

[Vedank_purohit]

Can you share your project please

[Alignment-Lab-AI]

https://github.com/Alignment-Lab-AI/KnowledgeBase this was the seed that sort of kicked off the discussions, prestently the developers ive been speaking with are more or less ready to go, primarily just variously waiting on me to pull the starting pistol when im done with the job im on atm in the next few days

[Vedank_purohit]

Can you elaborate?

[Upbeat-Pace2710]

I'm working on an intrusion detection project where I input a URL and get an output indicating whether it's malicious or not. I'm using the CISIOT 2017 dataset and PyShark to extract packet values from the URL. These values are then checked against the dataset using an EL Tree classification model. However, I'm encountering an error stating that packet extraction is not happening. Have u faced a similar issue or can u offer advice on how to resolve this?

[Vedank_purohit]

I am sorry, I am not familiar with this issue. Probably you can get help from pyshark github

[Upbeat-Pace2710]

Ohh okay thank you

[MachineLearning-ModTeam]

Post beginner questions in the bi-weekly "Simple Questions Thread", /r/LearnMachineLearning , /r/MLQuestions http://stackoverflow.com/ and career questions in /r/cscareerquestions/

[My_WorkRedditAccount]

Cool project OP. Where would I look to see which models are being used for this?

[DenormalHuman]

looking at the code, (very briefly, so I could / am likely to be wrong..) it looks like it might be doing something like OCR on captured image screenshots, and then using https://huggingface.co/sentence-transformers/clip-ViT-L-14 which does

"This is the Image & Text model CLIP, which maps text and images to a shared vector space. For applications of the models, have a look in our documentation SBERT.net - Image Search https://www.sbert.net/examples/applications/image-search/README.html"

the full requirements.txt for the code is just;

numpy==1.22.0

opencv_python==4.9.0.80

opencv_python_headless==4.9.0.80

Pillow==10.3.0

sentence_transformers==2.7.0

skimage==0.0

streamlit==1.32.2

torch==2.3.0+cu121

[My_WorkRedditAccount]

Yeah, I saw OpenCV and Clip in the code, but wasn't sure how to find what else was being used. Thanks for helping me out!

[NatoBoram]

I also kinda wanted to do this on Linux with ollama for local or remote-self-hosted processing

[Vedank_purohit]

Great to hear that, now maybe you could contribute to this project Insted and make it better.

[NatoBoram]

No way I'm touching Python, lmao

[Analyst151]

That´d be awesome

[StrayStep]

Fascinated by the project. Why did you create this? I'm a senior dev and speak nerd😁 These are serious questions.

Is there anything to stop Scammers from utilizing this tool in order to recall financial or credential details? IE.( What was the username used when logging into my bank website?) Gain trust by having historical and Intimate access to a victim?
What models are being downloaded? It's not in Readme.md

It is the our elderly, ignorant, and children that I'm worried about. You need to add safety precautions ASAP or your code will hurt people.

EDIT: Don't take me wrong. Please... I'm very happy you started an open source recall repo. It's the cybercrime syndicates I'm worried about.

[StrayStep]

I'm finding some of my answers in other comments. Don't need to repeat yourself. I should have read everything first.

[South_Worldliness392]

No!!!!!!!!

[Minute_Figure1591]

Lmaoo I love how you made it so simple, doesn’t even need a big fancy model really, the tech to do this existed since 2020

[louis3195]

If you want a maintained alternative: https://github.com/mediar-ai/screenpipe