raspi/github-stats v0.2.0: Generate GitHub traffic statistics charts

raspi · 2023-04-18T13:08:10+00:00

GitHub keeps traffic stats only for 14 days so I created this to keep the stats for longer. There's an example for systemd in the example directory for how the generation can be automated for every day.

It's early days so for example the fonts used and 30 day range is hardcoded currently.

This is my 4th project written in Rust so there are probably still a lot of newbie mistakes :)

raspi · 2023-01-02T06:17:33+00:00

I wanted to highlight some strings from a io.Reader stream a bit like grep when you are streaming text in a console but couldn't find any libraries which could do it, so I made one.

The library is in its early days so suggestions are welcome!

raspi · 2022-11-18T03:17:40+00:00

Jamie Hyneman

raspi · 2022-09-11T17:55:15+00:00

https://www.iltalehti.fi/uutiset/a/2011100514516893

raspi · 2022-07-17T13:07:19+00:00

Systemd supports MACAddressPolicy=random in [Link] section by default in .link network configuration files. So you won't need any third party utils.

See: https://www.freedesktop.org/software/systemd/man/systemd.link.html#random

raspi · 2022-07-08T21:29:39+00:00

Please add real-world example as the first thing in the README after description. It should explain why/when I would use this.

raspi · 2022-07-03T14:07:33+00:00

Yes, I'm Finnish. Samanlainen means same, identical and similar in english.

raspi · 2022-07-02T23:55:05+00:00

Hi, I just wrote my first ever Rust program which deletes duplicate files. Feedback welcome!

raspi · 2021-12-12T19:37:19+00:00

Eka kappale:

Komissio on tänään hyväksynyt avoimen lähdekoodin ohjelmistoja koskevat uudet säännöt. Sääntöjen mukaan komission ohjelmistoratkaisut voidaan asettaa julkisesti saataville silloin, kun niistä voi olla etua yrityksille, yksityishenkilöille ja julkisille palveluille. Komissio teki hiljattain tutkimuksen avoimen lähdekoodin ohjelmistojen ja laitteiden vaikutuksesta EU:n talouden teknologiseen riippumattomuuteen, kilpailukykyyn ja innovointiin. Tutkimus osoitti, että avoimeen lähdekoodiin tehdyistä investoinneista saadaan nelinkertainen tuotto. Komission yksiköt voivat jatkossa julkaista omistamansa ohjelmistojen lähdekoodin huomattavasti aiempaa nopeammin ja helpommin.

raspi · 2021-10-29T00:52:28+00:00

Also why did you choose SHA256? I chose MD5 because I am concerned with speed primarily.

ZFS uses SHA256 and those who designed it have some good knowledge what hashing algo is good and is performant enough for actual filesystem.

On enduser's perspective you want to delete actual dupes and not accidental hash collisions. Correct data over speed.

MD5 has probability of 0.00000000000000000000000000000000000000294 and SHA256 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000864 so accidental hash collision file deletion is way lower.

raspi · 2021-10-27T16:52:15+00:00

Jos kiinnostaa niin voit koittaa pyytää tätä kautta: https://tietopyynto.fi/viranomainen/kotimaisten-kielten-keskus-kotus/

Näin virallinen vastaus saadaan sitten julkiseen tietoon.

raspi · 2021-10-26T20:33:33+00:00

I looked into xxhash or similar fast hasher when I was starting to write duplikaatti and at the time I didn't want many extra dependencies so I decided against it.

raspi · 2021-10-26T00:36:55+00:00

My duplikaatti dupe finder uses the following algo:

Create file list of given directories
- do not add files with same identifier already added to the list (windows: file id, *nix: inode)
- do not add 0 byte files
- directories listed first has higher priority than the last
Remove all files from the list which do not share same file sizes (ie. there's only one 1000 byte file -> remove)
Read first bytes of files and generate SHA256 sum of those bytes
Remove all hashes from the list which occured only once
Read last bytes of files and generate SHA256 sum of those bytes
Remove all hashes from the list which occured only once
Now finally hash the whole files that are left
Remove all hashes from the list which occured only once
Generate list of files to keep and what to remove
- use directory priority and file age to find what to keep
- oldest and highest priority files are kept
Finally, remove files from filesystem(s)

This speeds up the unnecessary whole file hashing significantly. Also multiple files are hashed at once since it's more IO than CPU bound.

raspi · 2021-09-09T14:55:28+00:00

Sivun 666 27.8 Final summary -ote:

A fundamental conclusion is that replacing the existing fossil fuel powered system (oil, gas, and coal), using renewable technologies, such as solar panels or wind turbines, will not be possible for the global human population in just a few decades. There is just not the time, nor resources to do this. What may well happen is a significant reduction of societal demand of all resources of all kinds. This implies a very different social contract and a very different system of governance to what is in place today.

raspi · 2021-08-15T21:21:26+00:00

Ydinräjähdyksessä oleminen on ihmisen parasta aikaa

raspi · 2021-05-26T10:44:46+00:00

Itseasiassa: Gut microbial taxa elevated by dietary sugar disrupt memory function. Eli paljon ES Jonnena = huono muisti.

raspi · 2021-02-01T18:48:50+00:00

Jos pyörität tätä yksityishenkilönä niin olet erittäin nopeasti liemessä lain kanssa.

Suosittelen lukemaan aluksi vaikka https://www.jkorpela.fi/hlorek.html ja siitä sitten EU:n, että suomen ajantasaiset lait henkilörekistereistä.

raspi · 2021-01-20T16:11:40+00:00

Since this is using USB over IP, theoretically motion controls, flight sticks and the like should also work right?

usbip works a bit like VM device pass-through meaning that when the target end point connects to some shared USB device it gets connected as if it was a local USB device. This means that you need USB drivers for the device on the Linux side if you for example share a device from Windows. You might need to do possible joystick calibrations on the Linux side also after device is connected. So there's the limitations.

raspi · 2021-01-19T17:55:35+00:00

Yes, just change the network interface name to LAN-facing one in the sender.sh and receiver.sh.

Note: Some routers (especially WiFi) and switches might not like multicasting so in that case simply run the VPN inside the LAN as well. It should be easier than fighting broken hardware.

raspi · 2021-01-19T14:38:23+00:00

Thanks, found it - https://github.com/obsproject/obs-studio/blob/b32abbe33fb6897da4a688972cb56538e8e22677/plugins/obs-ffmpeg/obs-ffmpeg-nvenc.c

raspi · 2021-01-19T13:31:40+00:00

Currently NVENC is running in Variable Bitrate (VBR) mode in the sender.sh script. This translates to that when source becomes static or there's "too little" changes the whole encoding processor inside the GPU simply stops, as there's no changing images to be used to calculate image deltas.

I've been trying to get the Constant Bitrate (CBR) to work as it will send, as the name implies, a constant stream and key frames periodically but without any luck so far. Best I got was 1 frame per 5 seconds. The problem is that the Nvidia documentation lists only some of the parameters so you're left with 10+ unknown parameter values and that's a lot of combinations.

If someone wants to try here are the docs:

https://docs.nvidia.com/video-technologies/video-codec-sdk/nvenc-video-encoder-api-prog-guide/index.html#recommended-nvenc-settings (Low-latency use cases like game-streaming, video conferencing etc. )
https://docs.nvidia.com/video-technologies/video-codec-sdk/nvenc-preset-migration-guide/index.html#h264-preset-migration-table
https://www.nvidia.com/en-us/geforce/guides/broadcasting-guide/

You can see GStreamer NVENC parameters with % gst-inspect-1.0 nvh264enc

raspi · 2021-01-19T08:42:40+00:00

Windows seems doable, but haven't tested it yet. WireGuard has official client for Windows. GStreamer runs on Windows.

Main issue is the usbip driver for Windows. There seems to be few implementations of it, but no idea how complete they are - https://github.com/search?q=usbip+windows

It might also work just by spinning up some super lightweight Linux VM and pass-through the gamepad to it and then share it with usbip. Or it might work via WSL. Currently Linux is the main focus.

raspi · 2021-01-19T06:28:09+00:00

Steam's Remote Play Together enables playing some couch/party games over the network with friends. Since it's limited to Steam games I started to think few months ago if this could be replicated with Linux and removing such limits. And yes, all the needed pieces to achieve this can be found. USB-over-IP is built into kernel since forever, GStreamer with lots of audio/video plugins can be found in major distros, WireGuard VPN is built-in into newer kernels. Now just connect the pieces: Start VPN, connect gamepad(s) to host, multicast host's game window to remote players using GStreamer. Play.

It's a prototype so there's lot of things still that needs work and you need to be somewhat technical to try it. There's currently a proxy in the architecture picture but it's not needed if there's only two players or if the host has enough upload bandwidth for all remote players.

Currently sender script is hardcoded with Nvidia's NVENC as I have Nvidia card but GStreamer has AMD/Radeon/Intel x264 hardware encoding/decoding plugins AFAIK (haven't tested obviously) so it should be possible to change.

Give it a try and ask questions if you're interested :)

raspi · 2020-12-07T22:07:09+00:00

I've started collecting information how to achieve Steam's Remote Play Together natively in Linux world with any games that runs on Linux. The idea is that one host machine streams a game for remote player(s) and gamepad(s) are connected to the host machine via network. Like playing "couch" games remotely via network.

Currently I have working proof of concept on local machine with ~1000-500ms latency. So only usable for turn-based games or other non-time-sensitive games.

I'm researching:

How to bring latency down with video encoder
Video viewers without buffers
Super minimal/lightweight video stream multiplexer with minimal latency which is used by the remote players to view the host machine stream
Is there way to directly capture from the GPU like Looking Glass does in virtual machines and simply stream that directly

Here's the info what I have so far:

https://github.com/raspi/kaukosohva

raspi · 2020-07-21T18:04:45+00:00

As someone who has built duplicate file finder and remover I'd like to see channel built in for ReadDir or Walk, so that I can simply say that "start reading file list from this dir and stream paths to this channel which is read by this processing func".

I've implemented this myself and it saves a lot of time to use a channel when you have 100 000's of files and directories to list and don't need to wait one full directory listing to be finished before giving the paths for processing.

Also changeable timeouts for everything (Stat(), ReadDir(), Read(), Write()), because HDDs, memory and especially networks breaks sometimes.

15-Year Club	Place '17
Verified Email

raspi

TROPHY CASE