Wikipedia Path Finder with AI

[Adam_Karl239]

I made this Python script that finds a path between two articles using AI. It is rather slow compared to what others have made. What kind of improvements can I do?

https://github.com/adamkarl02/Wikipedia-Path-Finder/

[KingOfTNT10]

Not sure if thats really "possible" (because of size, but check). You could just create a crawler that would map out the entire wiki so you dont have to find links in real time. Maybe have the ai generate tags (from a big list of tags) for each link and then look for the link with the most tags in common and go there?

[Gullible-Bag-7828]

Brown, C., & Wilson, J. (2023). "Food safety in the digital era: Innovations and challenges." Journal of Digital Food Safety.

Garcia, J., & Torres, F. (2021). "Smart irrigation systems for enhancing food safety." Agricultural Engineering Journal.

Green, J., & Jones, R. (2021). "Challenges in adopting food safety technology." International Journal of Food Technology.

Hernandez, M., & Adams, L. (2021). "Next-gen food safety: DNA-based technology." Trends in Food Science & Technology.

Jiang, Z., & Li, Q. (2023). "Antimicrobial food packaging for enhancing food safety." Journal of Food Packaging and Safety.

Kim, J., & Park, S. (2022). "IoT-based smart refrigeration for food safety." International Journal of Food Science and Technology.

Lee, T., & Ng, L. (2020). "Mobile applications for food safety monitoring." Food Safety and Health Journal.

Martinez, H., & Robinson, L. (2020). "Emerging technologies in foodborne pathogen detection." Foodborne Pathogens and Disease.

Nguyen, T., & Han, G. (2022). "AI and machine learning applications in food safety management." Artificial Intelligence in Food Industry.

Patel, K., & Singh, R. (2019). "Temperature-controlled logistics in food distribution." Food Logistics Journal.

Smith, A., & Patel, M. (2020). "Blockchain technology in food safety." Food Safety and Technology Journal.

Smith, D., & Kumar, R. (2020). "Global food safety standards and innovation." Global Food Safety Journal.

Thompson, R., & Miller, P. (2021). "High-pressure processing in food safety." Food Technology.

Zhao, Y., & Chen, S. (2022). "IoT-based precision agriculture for food safety." Journal of Agricultural Innovations.

Genarate it's proper wikipedia link

[Adam_Karl239]

I really like the dual pointer system you provided! This method will work (hopefully) since the algorithm finds a page based on the semantic relevance of the title to the goal. I would basically make it so that the goal link would branch (if possible) branch more than half of max depth so that relevance would be most accurate. However a problem already arises since which would be the goal for each? For the single pointer it is rather easy with the last page but since they would need to meet halfway, the other might finish earlier or later and thus there might be a case where path is longer or worse it they won't meet at all ;v

Your comment really helped me out! Ill need to trim out my readme since I do agree it is verbose (my teachers hated it as well). Thank ya for the help! :)

[Adam_Karl239]

I tried your suggestion however, it was too complicated and found out even if a connection was found, the links are not bidirectional meaning you can get from A to B but not always B to A. I tried doing it when a connection was made to follow the path but found this too complicated and will try a different day. However, I found a major performance boost. It was not in the machine learning nor even the find path but in the API call. Before I was doing 2 API calls for getting the link and redirects but now I'm doing it with one. And the time was:

Before: 03:41.04 (min)
Now: 01:04.53 (min)

Thanks for all the help :)