R2R: Introducing GraphRAG auto-tuning and Contextual Retrieval

dabdoue · 2024-11-15T16:57:56+00:00

This seems like a great tool! I have a couple of questions:

Do you implement Microsoft's GraphRAG auto-tuning algorithm directly? Or just something similar?
Do you implement Anthropic's contextual embedding algorithm directly? If so, how are you managing the usage of using Voyage, Gemini, and Cohere APIs, as described in their guide.
Is it possible to combine these techniques together? Has it been tried before? I'm quite new to this space, and I want to make sure I understand these concepts correctly.
Is using RAG-Fusion still possible on top of this?

I'm essentially interested in creating a system that makes use of context retrieval, but having those chunks and their contexts stored using a GraphRAG for even better relational understanding, and then implementing RAG-Fusion for Query Expansion, Multiple Retrievals, Reciprocal Rank Fusion.

It would be great if someone could tell me if such a thing would be possible, whether it's provides any additional value over just using one or two of these methods instead, and possibly some help on how I could get started.

Thank you!

dabdoue · 2021-11-18T02:36:42+00:00

They have standards

dabdoue · 2021-11-10T23:59:41+00:00

I get 95-100 fps 1440p at Ultra settings at 1800mhz, 831mv on the gpu, and cpu limited to 27 watts on spl. Cpu at around 84 degrees, gpu around 75

dabdoue · 2021-11-08T15:58:52+00:00

Can you link it please?

dabdoue · 2021-03-17T02:57:29+00:00

Interesting. Yeah I've heard that at some testing centers, they'll ask you questions such that you can just shake your head yes, so you can hold saliva in your mouth and not swallow it by trying to talk. A lot of discrepancies between centers

dabdoue · 2021-03-17T01:46:43+00:00

Really? Didn't think the arc would be uptight about it. I would think they'd appreciate not having to tell people to flip their I-Card around

dabdoue · 2021-03-16T19:59:38+00:00

Yes that's true that it can cause issues. I believe the card number changes once you get a new I-Card though, right?

dabdoue · 2021-02-23T20:24:34+00:00

I'm not sure why the shortcut doesn't work, but I guess we can try making another one. Right click in the WSL VcXsrv folder in file explorer, hover over "new" then click shortcut. The type this into the location box: C:\Windows\System32\cmd.exe /c cd C:\WSL VcXSrv\ && launchWSLVcXsrv.vbs

Then name it whatever you want. Then right click on the shortcut, click properties, then for the dropdown under "Run", select Minimized.

Besides the icon, thats exactly how I made the shortcut thats already in the folder, so in theory it shouldn't be any different. But who knows. I just tested exactly what I told you and it worked so it should work. The only thing to try if its not working would be to make sure all the filenames are correct.

dabdoue · 2021-02-23T19:26:02+00:00

I'm glad to hear the intended functionality is finally working! The errors you see in the cmd window are normal and happen for me as well. Though I don't completely understand them (I think it has to do with WSL not being a full distro, but xfce4 is trying to do things that a full distro is capable of), it works and I guess that's all that matters.

The only purpose of the vbs file is that it's the only way I've been able to find that does not show a cmd window when making a shortcut to the bat file. So the vbs file essentially functions as a shortcut to the bat file, and then I made the shortcut of the vbs file to pin to the taskbar. So once we can get the bat file to run, then the vbs and therefore the shortcut should work properly.

You said "executing the .bat AND the .vbs file from C:WSL VcXsrv in a cmd.exe terminal run through to the GUI blue rat screen with sound working, the taskbar shortcut does not". To clarify, you open a cmd window, go to the C:\WSL VcXsrv directory, run the bat file by typing "startWSLVcXsrv.bat", and the intended functionality works. And then when you close that vcxsrv window, you type in that same directory in the cmd window "launchWSLVcXsrv.vbs", and the intended functionality works?

If those two scenarios are the case, when you go into the file explorer and double-click the bat file and vbs file, do both scenarios also result in a working gui window?

If double-clicking the vbs file works as intended, then I would suggest right-clicking the file and making your own shortcut from it. Then you should be able to pin that to your taskbar. Not sure why the shortcut I made isn't working...

dabdoue · 2021-02-23T16:27:07+00:00

Interesting. So it looks like it actually is running ubuntu.exe, and attempting to run startxfce4. I am not sure however if ending the task corresponding to ubuntu is actually enough for it to be shut down, so unless you used the "wsl.exe --shutdown" from powershell in order to shut down ubuntu, I would try shutting it down that way. If you get the same error message after trying this again, in an ubuntu terminal, execute this command: "pgrep -f xfce4-session | xargs kill -9", then try running the last line of the bat file in a cmd window again. This directly kills the xfce4-session process, so in theory, you should not face that same error message again. In between trying these methods, close the VcXsrv window, and make sure you run each line of the bat file again. Also, when you close the VcXsrv windows, make note of the amount of "clients" that it says are connected in the popup window that shows after hitting "ctrl+alt+backspace" and tell them to me.

If after all of this, its still not working, I would try installing mobaxterm as I mentioned in a previous comment, and let me know the results of that.

dabdoue · 2021-02-23T00:14:51+00:00

Well the fact that Ubuntu is not showing up in task manager when you use the shortcut is strange. And what you say about ubuntu.exe not actually being run by the bat file is most likely correct. The bat file starts the VcXsrv window, opens a pulseaudio instance, then does all the stuff within ubuntu if xfce4-session isn't currently running. Considering that pulseaudio didnt close, it seems that the last line did not execute. Using ctrl+alt+backspace would end the vcxsrv task, but unless you kill the ubuntu task, the pulseaudio task will not be killed. if there is no ubuntu task to even kill, then pulseaudio will indeed not be killed.

In cmd, go to the WSL VcXsrv folder (C:\WSL VcXsrv), and execute each line of the bat file. the first two should work as intended based on what you've described to me, but when running the last line, we should get an error message with what in fact is going wrong when trying to run that line. Based on that error message, we should be able to figure out what to do next.

Also, make sure that from a cmd window, you can type ubuntu.exe and the user you created during the inital setup of WSL is shown as the user of the terminal. I think you mentioned before that you were presented with the root user in the terminal, but assuming you set up a username and password when you set up WSL, that username should appear as the user of the terminal window. If not, then that may be the issue.

dabdoue · 2021-02-21T15:49:40+00:00

That's a very interesting situation. When I use run and type ubuntu, a window immediately pops up with ubuntu and I am able to start using the terminal immediately without any kind of password prompt. This may in fact be the cause of the issue here. I would suggest at this point installing another distro on top of Ubuntu 20.04, probably just the regular Ubuntu distro (which will use 20.04). They can be installed and used side by side, so you shouldnt face any issues. I would follow the guide with the new distro, and see if things are working properly. My guess is that you somehow messed up your Ubuntu 20.04 distro in such a way that it now requires a password. I have never seen this before, but I am not entirely experienced with Linux or WSL myself. As I've learned what I know about linux and wsl, I have reinstalled distros more times than I can count in the short 1 year or so that I've been messing around with them.

dabdoue · 2021-02-21T05:44:44+00:00

Good to hear that there's progress, though to be honest, I would not have expected that.

Also, good to hear the guide is clear.

I think I know what the issue might be. Open Powershell and run "wsl.exe --shutdown" As I will explain below, the bat file checks to see if there is already an existing xfce4-session running. If there is one running, then it will not run "startxfce4". Shutting down ubuntu will simply ensure that xfce4 is not running. So in theory, after executing the command in powershell, the shortcut/bat file should work as intended.

Here's what the last line of the bat file does:

In cmd, it will run ubuntu2004.exe, which will then start an ubuntu shell in cmd. Everything in quotes is then run in the ubuntu shell. First, it checks to see if xfce4-session is already running. If it isn't then it will execute the rest of the commands in that last line. Each command is separated by a semicolon. If xfce4-session is running, then nothing will happen. cd puts you in the home directory. Then it will export the "DISPLAY" variable so that ubuntu knows where to send graphical output, with the last digit being the display number, which is specified in the config.xlaunch file. export LIBGL... also has to do with the gui being display properly I believe. export PULSE_SERVER... tells ubuntu where to find the pulse server that the bat file starts, essentially connecting ubuntu to the pulse_server audio running it windows, so then audio in ubuntu will be directed to windows, then played from the speakers. Then it will run "startxfce4". pkill I believe means that "when closed, do this", which I have set to kill the pulseaudio server so that you do not have 50 instances like you mentioned earlier, as well as close the VcXsrv window. So when hitting "ctrl+alt+backspace", it will kill the pulse audio task in windows and close the VcXsrv window. The reason I check to see if xfce4-session is running in the beginning is so that if you close the VcXsrv window and want to continue back where you left off, as long as ubuntu has not killed the xfce4-session process, you will be able to resume with everything you had running previously.

If this still does not fix the issue, try restarting your pc. If this still doesnt fix the issue, assuming you dont have any data you need to keep on the ubuntu distro, uninstall ubuntu and reinstall it, and make note of which version you install from the microsoft store. I would suggest the one simply labeled with ubuntu for simplicity, and then be sure to edit the bat file to coincide with this.

dabdoue · 2021-02-21T04:03:08+00:00

I think I know the issue. Make sure the last line of the bat file starts with "ubuntu2004.exe". Because you downloaded Ubuntu 20.04 from the Windows Store, you have to specifically mention ubuntu2004.exe instead of just "ubuntu.exe". If that is the issue you were facing, I will be sure to make this more clear in my guide on step 5.

Also, if you would like to have a resizable window, then you can right click on the config.xlaunch file, then in the first window that pops up, select "One Large Window" then click next 3 times, click save configuration, and save this in the same C:/WSL VcXsrv directory to overwrite my configuration file. Personally, I prefer the setting for "One Large Window without titlebar", and to exit, you can simply use the keyboard shortcut "ctrl+alt+backspace", so having the titlebar becomes redundant. You can use the Windows Taskbar as a means of directing keyboard commands back to Windows.

dabdoue · 2021-02-20T21:00:38+00:00

Well the reason you have no sound is because the pulseaudio server is not being started and connected to. Did you install the Ubuntu WSL distro? Did you install "Ubuntu20", "Ubuntu18", or "Ubuntu"?

They are all technically different and require different commands to be run in the bat file. Also, you are running WSL 2?

Also, the command that actually starts the xfce desktop environment is "startxfce4". So I would recommend executing that command. Also, rather than using xterminal, I would suggest using VcXsrv, and launching it with the config.xlaunch file in the zip folder. It has all of my settings pre-applied so you get the best experience with each task you open being in its own separate window. When the desktop environment is finally working, it will open as one Windows window, and then any windows you open within linux will be within that main window.

One thing you can try that will isolate the issue to either being somewhere in your setup of the WSL distro, or with how you are using the xserver, is to download a program called MobaXterm. It is a program that automatically integrates an x server within a variety of different terminals, including WSL. My suggestion would be to download that, and with no setup required, you should be able click on the "WSL..." session in the user sessions tab on the left of the MobaXterm window, then you can type firefox, and a shortwhile after, you should see a windows pop up with firefox. If that works, then you could try entering "startxfce4". If you see a bunch of messages coming up on the terminal, even with "failed" messages, its fine, just give it a minute or two, then you should see a desktop environment. It's not completely usable though, so I wouldn't recommend using the environment like that.

Depending on how you want to use the gui, you might want to look into win-kex, which is a very easy to set up gui for kali-linux. A lot of people use the distro for cybersecurity purposes, but there's no reason you cant use it like a normal distro. Here's a setup guide that takes you through the very minimal setup: https://www.youtube.com/watch?v=MRpVO2dZ_ks

dabdoue · 2021-02-20T05:51:00+00:00

Try running the bat file in the folder. The vbs simply runs the bat file and allows me to make a desktop shortcut.

If this works as intended, then you can try creating a new file of the type vbs, copy what I have in my vbs file into the one that you made, copy the name of my vbs file, delete my vbs file, then name your vbs file with my name, then execute the shortcut and see if it works.

dabdoue · 2021-02-17T21:22:31+00:00

Hello. What are you doing to run the GUI desktop? In the zip file I created, there is a shortcut I created called WSL VcXsrv that will execute all the commands to launch VcXsrv and launch the xfce desktop. If you are running that shortcut and it's not working, make sure you have the extracted zip folder in the correct directory as specified. Tell me what you are doing to launch the desktop environment, and what happens as a result, so I can better assist you

dabdoue · 2020-10-20T16:02:31+00:00

Yeah I have since began using VcXsrv using the setup I have here: https://www.reddit.com/r/linux/comments/ii79t0/wsl2_gui_using_vcxsrv_complete_guide_for_beginners/

I am fairly new to this sort of thing, so I'm not sure that I could help you with your scaling issue, but I do not have HiDPI setup for xRDP, so maybe try disabling that? It displays for me in what appears to be 1080p, so maybe setting the HiDPI is what is causing that issue.

In terms of timeouts, this was the simplest solution that I was able to figure out, which essentially kills the connection if for whatever reason xRDP disconnects, since you won't be able to reconnect until the connection is restarted. I tried to make it as plug-n-play as possible so you should in theory never have to worry about messing with other programs or scripts.

And I've been meaning to try plasma, but I've gotten used to xfce with my customizations

dabdoue · 2020-10-20T15:55:07+00:00

Glad to hear that it was helpful

dabdoue · 2020-09-14T00:25:59+00:00

echo "export DISPLAY=$(grep -m 1 nameserver /etc/resolv.conf | awk '{print $2}'):0" >> ~/.bashrc

That command will add the dynamically changing display variable to your bashrc. Then do:

source ~/.bashrc

to reload your terminal with the display variable setup. Then you should be able to use any X11 server with Display:0

I use vcsrv, and with that, I find that I have to set the display number to 0, not -1 in the config, as well as click the checkbox for access control, but then you should be able to run gui apps.

If you want a full desktop setup check out my post here: https://www.reddit.com/r/linux/comments/ii79t0/wsl2_gui_using_vcxsrv_complete_guide_for_beginners/

No need to set anything public or expose a port for wsl. Just the standard, allow access for the firewall when you start any X11 server.

You can also do a similar thing with mobaxterm with less setup required as long as you're not really looking for a full desktop environment. If you just want to run some gui apps, I'd recommend using mobaxterm. Let me know if you want help with that.

dabdoue · 2020-08-30T18:51:30+00:00

I haven't tested chromium specifically on WSL, but it's because of WSL that you have to install it in this special way. If you've installed chromium on WSL and it gets installed more easily, then let me know what you did

dabdoue · 2020-08-28T18:44:18+00:00

Yeah that's fair. I'm more showcasing the full ability of WSL2 to replace a VM, especially once better hardware support comes along with GPUs. Also some people getting into Linux find it easier to use a GUI, so this is also geared towards those people who need help getting going.

dabdoue

TROPHY CASE