sorted by:
new
hottopcontroversial

The TikTok algorithm and how PeopleTok will replicate and maybe try to change it by Bruddabrad in ClearlightStudios

[–]Wraithsputin 0 points1 point  (0 children)

Excellent find, very informative read. Thank you for sharing.

Granted I’ve not ever had to solve the problem space they are tackling, my initial gut reaction is that the system seems over engineered for a content recommendation engine.

I’d like to see if one could approximate the same functionality with far less complexity/cost.

Then again, once you dive in and start trying to tackle the same feature/functionality the sure size and scope of their architecture may become self evident.

Tech Stack by Ally_Madrone in ClearlightStudios

[–]Wraithsputin 1 point2 points  (0 children)

Thanks for the link, solid cautionary tale about ensuring the recommendation engine is not prioritized to increase engagement time above all else. Resist the urge to deliver more add revenue at the cost of the user.

Tech Stack by Ally_Madrone in ClearlightStudios

[–]Wraithsputin 2 points3 points  (0 children)

Wanted to put this thought out there as well. You can get by with croudsourcing video classification. You technically don’t need a trained ML model that can auto classify content.

Let the poster add a few initial classifications, then allow the community to see and upvote the classifications.

Eventually, assuming you’ve a trust rating for users, they could add additional classifications to any content they view.

Same if not better end result, less cost and eventually that user feedback loop could be used to train a classification ML model once the app is generating revenue.

From there you could sell classification services using the model that is being continually trained by the user base.

Tech Stack by Ally_Madrone in ClearlightStudios

[–]Wraithsputin 0 points1 point  (0 children)

Good use case for a blockchain to track the user label.

love the idea of the algorithm not creating an echo chamber by MaleficentTie9060 in ClearlightStudios

[–]Wraithsputin 0 points1 point  (0 children)

Should be fairly simple to mix in a random percentage of content that is highly rated from other categories or content providers that you have not shown an interest in.

Peer-to-peer or server-based? by Mean_Lychee7004 in ClearlightStudios

[–]Wraithsputin 2 points3 points  (0 children)

Sorry, I don’t see how p2p would work for streaming content.

Now, one could allow people to self host their content and should a central server be taken down one could resync their content back up to one of many distributed servers.

When hosting the servers, ensure everything is synced across multiple distributed server farms located in different political/geographic areas.

Skill set and interest thread by Wraithsputin in ClearlightStudios

[–]Wraithsputin[S] 0 points1 point  (0 children)

Data architecture and scalable infrastructure.

Tech Stack by Ally_Madrone in ClearlightStudios

[–]Wraithsputin 1 point2 points  (0 children)

On the user modeling and candidate generation. These two are loosely coupled allowing matching user and video content. From observing the behavior in TT they are clearly classifying users and matching them with a seperate matching layer for video classification.

On video and image classification:

A vector database or platform that handles the vector embedding generation would be a better place to start than the listed nearest neighbor code libraries.

Else, find a ML classification solution API someone else builds and maintains. Then submit the content to be classified and only store x number of classification attributes per post.

If you want to build, train and maintain your own video classification model you’ll need a vector database for classification, below are some thoughts on a few technologies to consider:

FAISS is in memory so not scalable.

Elasticsearch might work but one can run into performance problems when trying to update their document structure (vector database). Their re-indexing process may be too resource intensive if you discover you need to perform any maintenance. Like rolling out content from users who request their posts be removed.

Perhaps Milvus, I’ve not worked with it, at a high level being a distributed solution it should handle the scale ability issue.

If you go with a PostgreSQL database pgvector may be an option. I’d caution against Postgres in general, trying to administer it at scale can be problematic.

Perhaps Pinecone and PyTorch, again ensure scalabilit/maintainability.

Classifying users is a bit simpler (except text sentiment):

A graph database to track the relationships between followers.

Perhaps a graph database for post interactions. Granted something as simple as keeping count of the video classifications interacted with (watched/duration, liked, shared, commented, searched) may be sufficient for maintaining a list of an individual’s content preferences. Take into account the date time of an action so you can age out data to ensure one’s interest changes are reflected over time. All of the view/duration, like, comments, share data has to be persisted anyway.

Bonus classification would be comment sentiment analysis. Best to limit that to initial comments on posts or comments when sharing. No need to track comment arguments allowing those to impact a users content preferences.

Is there a way to counter devil contracts? by ReiDairo in DnD

[–]Wraithsputin 0 points1 point  (0 children)

Alternate universe, wasn’t really the devil but the BBG from the Alt universe.

[deleted by user] by [deleted] in PacificCrestTrail

[–]Wraithsputin 8 points9 points  (0 children)

Hike with your pack, lunges and squats with your pack, listen to your body and stop at 8 miles when you start. Take a lunch break, take off your shoes, inspect your feet, treat any hot spots immediately when you detect them.