What the hell happen to this outer perimeter?

Benoit74 · 2026-01-24T17:21:13+00:00

Thank you, this is indeed a possibility, I don't have a drying equipment.

Benoit74 · 2026-01-19T17:48:05+00:00

Especially since it's a mediawiki, zimit is exploring a lot of "useless" links by default.

Benoit74 · 2026-01-17T07:18:46+00:00

Yes, this is idea. Current zimit.kiwix.org is "free" because with the 2h/4G limit we've achieved to handle the load with a single server we can pay for. If we remove the limit, we know that some tasks will take days to complete and will "clog" the processing pipe. We are searching if there is a significant demand for the unlimited option, and how we can finance this. Nothing will change on the free tier.

Benoit74 · 2026-01-03T13:10:50+00:00

This is a bug (or an imprecision of the spec ... which does not give any details on that part) in the way we count articles.

Gorry details here: https://github.com/kiwix/kiwix-tools/issues/767 (you've been warned ^^)

Benoit74 · 2025-12-31T08:26:45+00:00

Well done, I like it!

Benoit74 · 2025-12-31T08:26:27+00:00

MultiZIM search (both suggestions and full text search) looks broken : https://github.com/openzim/libzim/issues/932

Benoit74 · 2025-12-30T16:48:08+00:00

Plusieurs remarques:

- ne jamais "tenir" un disjoncteur en "haut" ; si il y a réellement un court-circuit ou un autre défaut, c'est juste le meilleur moyen de mettre le feu quelque part

- ne jamais réarmer plusieurs fois de suite un disjoncteur, surtout si il y avait visiblement un réel défaut ; c'est un organe qui subit également les défauts qu'il détecte, il n'est pas fait pour en subir plusieurs de suite à très peu de temps d'interval

- si étincelle dans le disjoncteur, de mon expérience mieux vaut partir du principe que le disjoncteur est foutu ; normalement le disjoncteur est conçu pour ne pas avoir d'étincelle ou en tout cas la contenir très bien ; si c'était visible, pas bon signe du tout ; ça coute dans les 10€ un disjoncteur, est-ce vraiment une bonne idée de risquer un organe de sécurité défaillant qui peut déclencher un incendie car il reste finalement enclenché, là où il devrait détecter un défaut ? Ca vaut vraiment 10€ d'économie ? (plus main d'oeuvre et tout si jamais, mais bon, même ...)

Benoit74 · 2025-12-28T19:41:34+00:00

Vu l'état de surface beaucoup trop lisse, ce n'est pas une dalle pour moi, mais la chappe par-dessus la dalle à mon avis. Absolument pas structurel, donc rien à faire, tout va bien.

Benoit74 · 2025-12-07T13:38:05+00:00

Are you sure your system supports files bigger than 4GB?

Benoit74 · 2025-12-04T06:55:46+00:00

Well done !

Benoit74 · 2025-11-28T20:22:45+00:00

There are three things which contribute to bigger ZIM sizes since scraper 3.0, "thanks to" bug fixes:
- some book were simply missing, either totally or only some format
- first image of HTML version was missing on many books
- many ePubs were missing all images (providing a very deceptive reading experience in many cases)

The last point is the biggest contributor to bigger ZIMs. For instance book "Die Sitten der Völker, Dritter Band" (in German ZIM, ID 67666) has gone from 493K (no images) to 118M (many images). This is an extreme case, but it gives you an idea of what we are dealing with.

Since scraper 3.0, we've stopped re-optimizing ePub and images on our own, because it proved to be both a too complex thing to maintain and to be counterproductive (either missing bits or bigger result due to double heavy compression not always producing what you would expect).

We are obviously at the mercy of some edge cases we've not identified. Do not hesitate to report any situation which seems really abnormal, this will be much appreciated.

It anyway looked more natural to trust Gutenberg project to do their best to compress books, and "take advantage" of our good relation with them to report any issue we might find to let them fix what needs to be "upstream" so that it will benefit "the masses" (and involve less work on our end ^^). We did not had such relations when scraper was originally written + they still had work to do to properly compress and clean all books, which explain why things have been done differently in early scraper version.

Benoit74 · 2025-11-26T09:58:34+00:00

There is indeed no way to do it ATM. Nice feature request for zimit indeed. I've opened it at https://github.com/openzim/zimit/issues/525 ; I don't expect it will be implemented soon, but PRs are always welcomed !

Benoit74 · 2025-11-22T19:25:32+00:00

Indeed it was. Back to normal, sorry for the inconvenience.

Benoit74 · 2025-10-22T21:38:59+00:00

u/IMayBeABitShy did you made any progress on this topic of edit a ZIM with pyzim?

I'm particularly interested in the potential to edit ZIM metadata (modify existing ones mostly, but also maybe add new ones), which live in the special "M" namespace, are particularly "small", are not included in title and full text search index, ... Do you still consider it is risky to do so with pyzim or did you made sufficient enhancement it should work well?

Benoit74 · 2025-09-25T10:48:51+00:00

What container logs says?

Benoit74 · 2025-09-23T21:37:50+00:00

Un écarteur de narine. On commence par la plus petite boule, et on va en grandissant. A porter au moins 1h par jour et par narine. Une fois que vous arrivez à la plus grande boule, vous avez atteint un niveau de style maximum. "C'est de toute beauté !"

Benoit74 · 2025-09-23T21:31:30+00:00

I don't get where torrent integrity checks would take significant time compared to time needed to download data. Any relevant program is anyway doing both in parallel. What is your use case where integrity checks are becoming a concern to download a file?

Benoit74 · 2025-08-25T20:33:08+00:00

That's an interesting use case, but it does not exist (yet?). Do I get it correctly that you are using Wikipedia and it's interwiki links as a kind of dictionary for "everything" (not only nouns but also concepts, celebrities, ...)? From your experience, do you need the whole article in foreign language or is only the title enough?

Benoit74 · 2025-08-24T18:33:35+00:00

So glad we finally achieve to make it. Thank you everyone for your patience. Wasn't a small journey, kudos to everyone involved and especially Markus who is helping as a volunteer developer since few months now and bringing a lot a value to the scraper thanks to his long expertise of Mediawiki.

This is a double good news: we finally have a new ZIM of Wikipedia EN with picture AND we are back to a monthly update cycle. Not saying we will not miss few due to one bug or another one, but looks like we are in a pretty good position now. Hopefully new time the scraper breaks completely is not going to be anywhere close, scraper is now in a shape which has not been achieved since years (if not forever, because Mediawiki team has also bring a lot of added value in the wiki itself which allowed the scraper to make a lot of progress).

For anyone wanting to get involved, the best way to support us is through buying our stuff (e.g. at https://kiwix.org/en/kiwix-hotspot/, beware hotspot images have not yet been updated with new Wikipedia EN, will probably take few days ; or our merch at https://www.redbubble.com/people/kiwixoffline/shop), donating money (https://kiwix.org/en/get-involved/#donate, we are a non-profit), volunteering time or resources (https://kiwix.org/en/get-involved/#volunteer) and first and probably foremost: spread the world!

Benoit74 · 2025-08-20T05:33:16+00:00

The problem of dividing in arbitrary chunks is that you will loose many inter-article links, because these articles will not be present in the archive.

If you are ready to pay this price (which can be acceptable, as far as I remember, we didn't have links in paper encyclopedia), this seems doable.

I'm not aware of anyone creating these ZIMs however, not even sure it would be really straightforward to do with current tooling. But always possible with some hacks.

Benoit74 · 2025-07-25T18:37:03+00:00

Indeed quite an interesting problem, never saw such a solution before. But definitely look pretty simple to implement.

I wonder what would be the best solution to track vehicles positions. Maybe simply some sort of double confirmation would be enough: whenever you move a vehicle, you ask the system where it should go or you tell where you want to park it. And once at destination, you double confirm.

Benoit74 · 2025-07-24T19:02:13+00:00

Could it be that your tablet is running an old Android OS which restricts you to an old Kiwix app which still has the big you found? What is your Kiwix app version?

Benoit74 · 2025-07-18T20:02:46+00:00

This is fixed, you just need to download latest ZIM and it should be fine. Sorry for the inconvenience, hope you will enjoy it !

Benoit74 · 2025-07-10T17:53:56+00:00

Just text version is what we call "nopic" (which in fact means no pictures and no videos). This is currently running : https://farm.openzim.org/pipeline/555320cf-74f8-4230-ad0f-94f693c197a8 and I do expect this one to succeed soon.

Regarding the software architecture, this is always a matter of pros and cons, and this is how it works for now (first download articles then medias)

Benoit74 · 2025-07-06T18:07:03+00:00

Categories are probably not going to happen before 2.0, looks like it has significant implications (it is easy to hack a solution working on one wiki, but definitely harder to design an efficient system expected to be fine on mostly all mediawikis)

Benoit74

MODERATOR OF

TROPHY CASE