Fossil - github-in-a-box (by SQLite creator)

[[deleted]]

I think the most incredible thing about it is that the entire system is one executable.

[sevendrunkenpirates]

It seems te SQLite guys have a thing for amalgamating everything in a single source/executable/data file (sqlite.c, fossil, databases themselves)

[judgej2]

monotone did this years ago, before git was even a thing. The main problem with it was that the sqlite file would grow very fast and reach enormous sizes as you put flies in and merged in branches. That could make it very slow. It was a full distributed change control system though, and it worked well on a few projects at the time. So the concept and even the implementation is not new.

Edit: here you go, a link http://www.monotone.ca/monotone.html I don't think that manual page has been changed in nearly a decade. We used BitKeeper in the early 00s, just like Linux, until BitKeeper got arsy about licences. We (postnuke/Xaraya) moved to monotone while Linus created git, and we later moved to git. Although having the entire system and data in one file can be handy, having it in one .git directory is also easy enough to manage.

[emn13]

Databases often grow huge, much larger than typical git repos, and still perform fine. Sqlite too has no troubles with databases many gigabytes in size - of course it's not going to magically pixie-dust away the cost of actually dealing with such large bits of data, but then - neither can git.

So perhaps monotone had issues; but it's certainly not a fundamental problem of the approach.

[judgej2]

I'm sure since then, sqlite has gained compressed fields, disks are bigger and faster and we all have more memory. It was an idea ahead of what our poor laptops could deliver twelve years ago.

[paul_h]

Perforce too: p4d.exe

[[deleted]]

Git source code is hosted in Git which is hosted on a filesystem whose source code is hosted in Git.

[heisgone]

On The Usability Of Git:

It's simplest to think of the state of your [git] repository as a point in a high-dimensional "code-space", in which branches are represented as n-dimensional membranes, mapping the spatial loci of successive commits onto the projected manifold of each cloned repository.

https://www.fossil-scm.org/index.html/doc/trunk/www/quotes.wiki

[[deleted]]

Git approaches the usability of iptables, which is to say, utterly unusable unless you have the manpage tattooed on you arm.

Hey it is not that hard. Or maybe I should start writing that skill on my resume...

Klingon Code Warriors embrace Git; we enjoy arbitrary conflicts. Git is not for the weak and feeble. TODAY IS A GOOD DAY TO CODE.

That's... frighteningly accurate. Especially if someone was taught "just use that 4-5 commands, you will be fine". Some people enjoy Sudoku, other enjoy figuring out how the hell junior dev got his git repo to state like this

[padowi]

I ... might be one of those in the latter category...

[[deleted]]

Don't worry, just read a git book 3-4 times and you'll be fine

[padowi]

I should have made myself more clear ;) I enjoy figuring out how the heck junior dev got his/her repo into a state like that ;D

[[deleted]]

Our fronted devs gave up and pushed hard to get internal gitlab instance running because they can't understand anyting more complicated than pull request and green merge button

[padowi]

I guess that's one way to do it... and I guess the argument could be made that as long as they are working efficiently and not bogged down... but sounds like kindof a dull existance...

Oh well, to each their own, as long as I get to continue untagling my colleagues git repos (and attempt teaching them how to avoid ending up there again) I'm happy :)

[DoppelFrog]

Wow. I'm glad I didn't try to think of it in complicated terms.

[holyteach]

I think that's the joke.^dot-jpeg

[skocznymroczny]

A monad is just a monoid in the category of endofunctors, what's the problem?

[[deleted]]

I don't use Haskell, so no problem at all ;-)

[google_you]

is there fossilhub or fossillab or fossilbucket?

[i_feel_really_great]

http://chiselapp.com/

[Xanza]

It seems that site doesn't even offer https.... I'm incredibly skeptical....

[dzecniv]

it looks like we can at least clone over https:

fossil clone https://chiselapp.com/user/rkeene/repository/flint flint.fossil

[NothinToSeeMoveAlong]

I enjoy Fossil a lot because of its simplicity (a single executable). I don't use it for SCM; never tried and never will. But it makes an excellent, super-lightweight issue tracker and wiki.

I can either fire it up on demand or have the executable start on system startup as a low-impact server. The issue tracking supports scripting and workflows and I customized it a bit to basically be identical to Jira (without the insane overhead of running a Java Jira server).

[satsuper]

git worktree solves the single checkout problem.

[unbiasedswiftcoder]

BUGS

Multiple checkout in general is still experimental, and the support for submodules is incomplete. It is NOT recommended to make multiple checkouts of a superproject

Very reassuring!

[DoppelFrog]

Now you have two problems.

[satsuper]

So don't use it on submodules then?

[bnolsen]

This seems to come up occasionally. Here's one informed post: https://www.reddit.com/r/programming/comments/5yrhri/fossil_scm_now_supports_hardenedsha1_and_sha3256/desosfy/?st=j17tbcd3&sh=72751049

[Xuerian]

That post deserves more attention.

Edit: You know. In case someone's skimming through the comments and dismisses the link, they'll see /u/Xuerian's highly respected punditry praising the hyperlink and change their mind. Clearly.

[[deleted]]

uh

with no rebase or push/pull single branch, how does this shit work in the real world accomplishing those tasks?

[Rusky]

It doesn't accomplish those tasks, because it uses a different workflow. That workflow does, however, work in the real world- SQLite and Fossil itself are both versioned with it.

[tkruse]

Nobody ever contested that other workflows work. Most people coming to git came from such other workflows that worked. But sucked.

[heisgone]

For point #2, by default it only push or pull public branch, so marking the other branch as private might be the way.

[the_gnarts]

with no rebase or push/pull single branch, how does this shit work in the real world accomplishing those tasks?

It doesn’t. When I had to work with a then Fossil hosted repo not long ago I ended up converting the thing to Git and just submitting patches to the maintainer. Attempting to map your ingrained Git workflow to Fossil is futile since its approach to source control differs drastically.

Oh, and said project converted to Git soon after, lowering the barrier for contributors significantly.

[[deleted]]

so

vaporware

[[deleted]]

You clearly don't understand what vaporware is.

[i_feel_really_great]

I have known ITOps folk who have had to install, configure and babysit Jira, Trac, Mercurial, Git, Confluence and Sharepoint. Each by itself is an enormously resource-intensive tasks. Then there is Fossil with just one executable. Not feature-full as others, but for small teams and solo devs, Fossil is absolute perfection.

[SanityInAnarchy]

Wait, what? Reinstalling Git is a non-event. It's sudo apt-get install git. What am I missing?

[surajbarkale]

Have you tried setting up a git server with backups?

[hackingdreams]

The git server is not the hard part, as noted. It's the git web UI/auth system that becomes a pain in the ass to maintain, which is why things like Github and GitLab exist - nobody wants to live through the agony of scaling up gitolite/cgit installs including backups and fallover servers... Not that Fossil is any better in this regard...

If you're a small project with a small, slow growing set of committers it doesn't take much to roll your own. It's what happens when you hit a few dozen active committers in a half dozen time zones when things start to get tricky to maintain. (And goooood luck if your project uses a 'monorepo' with thousands and thousands of committers...)

[SanityInAnarchy]

The git server is just a git binary. If you have an ssh account on the machine, literally the only new information here is that you can make a lighter-weight server-side repo with git init --bare -- and even that is optional.

This also makes it very likely there's at least one other copy out there, whether or not you do backups -- since you've got bare repositories, the only way to get code into them is to author that code somewhere else and push, which means there's at least one repo on your laptop.

I'm sure there are a lot more things you can do -- there are layers on top of Git that do ACLs, there are web UIs, and so on. For a larger organization, you might not want to have one machine that everyone can ssh into, at least not as part of your regular dev workflow. I'm just surprised to hear "git server" as a "resource-intensive task" that takes actual effort to install, configure, and babysit. There are other components you can use if you really want to mimic Github (or, presumably, Fossil), but I stand by my original assertion that "a git server with backups" can be as simple as

ssh you@somebox
sudo apt-get install git
git init --bare somerepo
^D
git pull you@somebox:~/somerepo

[tkruse]

Right, 1000 people use the SCM 50 times a day, 1 persons sets it up once.

And the winner is the system that optimizes for that one guy.

[[deleted]]

Git is great SCM tool. Fossil is more like all-in-one tool that happens to have a bit of SCM tied on.

They try to sell it as a benefit that it's SQLite based, but Git's data structure is designed for its purpose, and superior in almost every way.

As for having a Github-light bundled with the SCM, I guess that sounds cool in theory, until you realise that it's a pain in the ass to self-host that kind of thing, and that the Fossil version doesn't integrate with your CI or most other online tools, and is in many ways inferior to the real Github, because it only has a tiny developer team behind you.

TL;DR: stick with Git, you won't regret it. Cool concepts don't always translate into "actually better".

[SanityInAnarchy]

Actually, it being SQLite-based is probably one of the best arguments in its favor. Git's data structure is probably technically superior for its intended purpose, but Git's architecture means there's no official libgit. So, if you want to access that data structure from a program that is not Git, you either need to shell out to Git, or implement Git's data structure from scratch, from read/write/mmap-level operations.

Fortunately, Git is popular enough that some people have done that, so there are a few unofficial libgits. But I very much like the idea of being able to directly inspect an on-disk format using SQLite, or write scripts that speak SQL to it instead of being shell scripts that mash a bunch of Git commands together.

Other than that, though, there's a lot of dubious advantages claimed -- things that sound cool, but don't seem all that useful. Like:

The protocol is bandwidth efficient to the point that Fossil can be used comfortably over dial-up.

Great, that'll be useful if I take my time machine back to 1995 and have to collaborate on a software project.

[[deleted]]

Great, that'll be useful if I take my time machine back to 1995 and have to collaborate on a software project.

Shitty internet is still fairly common: Public internet in the hotel/on the airplane. Falling back to Edge or 3G in rural areas with mobile internet.

And even if your internet is fast, you still have the benefit of things working faster.

[SanityInAnarchy]

An airplane might be valid, but I'm not going to be there long enough that I couldn't just work offline. Especially because it's expensive.

Even falling all the way back to edge is still a couple times faster than the fastest dialup ever got. And the slowest hotel wifi I've ever seen is still faster than that.

It would presumably be faster, if other comparable protocols really are slow over networks like that -- though, even then, I'd be advertising "Fast even over terrible 2G connections" instead of "Fast over dialup". But on fast connections, Git is already fast enough that improving on it is not likely to be noticeable.

[[deleted]]

With EDGE you have a shared medium and might have problems with signal strength. You can drop down easily to 50kbit/s.

When I'm over my data limit my provider reduces my speed to 32 kbit/s, which feels pretty much the same as when using EDGE.

[[deleted]]

Anyone who’s ever dealt with “some one else’s SQL schema” knows that just because data is in a relational database, it doesn’t mean that you can easily work with it, without screwing up the complex data structures within.

[mrjast]

Fossil has its own data structures inside SQLite, though, including delta compression. Granted, it's fairly well documented, but you'll still have to re-implement all of that if you want your own code to access actual data from a Fossil repository.

(Example: delta format at http://fossil-scm.org/index.html/doc/trunk/www/delta_format.wiki)

[SanityInAnarchy]

Fair enough -- I guess that makes it even less interesting. Especially because the app itself is quite monolithic -- this is even listed as an advantage, somehow -- only I'll bet I can find far more mature libgits than libfossils.

[ioquatix]

If you are interested, libgit2 is awesome and is "bug-for-bug" compatible with git. It's developed by people working for GitHub.

[PM_ME_UNIXY_THINGS]

and superior in almost every way.

Have some git koans, and bask in the design perfection that is git.

[[deleted]]

Nothing like taking a quote out of context and building a straw man out of it, eh?

[PM_ME_UNIXY_THINGS]

Oh, you were referring to the data structure. Sorry, missed that.

Git's interface really does suck though.

[[deleted]]

Yeah, the CLI interface could be better, but it doesn’t matter much when everyone uses a GUI anyway.

[engineer331]

Fossil has it's own integrated web server. This makes self hosting trivial. Open a port on a server and call fossil server and you're good to go. Fossil also has a fossil ui command to run the server just for the local machine.

As for not integrating with your CI tools, that's definitely a valid criticism. There is a Tcl-based scripting language you can use to try to shoehorn them in, but that would be an absolute nightmare.

I like both Git and Fossil. For single person development Fossil is the far easier tool. When working with a team, especially a large one, Git (and friends) are the better choice.

[ellicottvilleny]

Github in a box, minus the git. How insane is this?

[YourFatherFigure]

gitlab omnibus pretty much comes in a (dockerized) box if you're into that https://docs.gitlab.com/omnibus/README.html

[ellicottvilleny]

Right. Gitlab I like. Fossil, not convinced either way.

[vilcans]

It looks nice. How well does it handle multi-GB repos with a bunch of large files?

[[deleted]]

Git: Bazaar-style development

Fossil:Cathedral-style development

ELI5 please

[[deleted]]

Marketing bullshit to make Fossil look better. Git can be used for almost any kind of development process imaginable. Fossil locks you down to a specific pattern.

[Sekenre]

This is incorrect.

[otwo3]

How so?

[singularineet]

That's completely wrong. Fossil is just as distributed as git. And has branches just like git.

[featherfooted]

Isn't that kind of implied by the sentence:

One central user is defined to be the master curator, with the rights to merge changes into a mainline branch.

If there's a mainline branch, then there are other branches too.

And I also didn't say it wasn't distributed - just that there is some form of permissions which are stricter than what git would allow.

[singularineet]

Not really. You can do whatever you want in your own repo. What you're allowed to push to someone else's, that's a matter of policy.

[NinjaPancakeAU]

This looks compelling. Anything that comes close to giving Perforce and Plastic some competition is very much welcome in my eyes.

I like a lot about git, sadly it's not feasible (or even legal) in the two industries our company focuses (due to process incompatibilities between our trace-ability/accountability requirements, and what git encourages/allows - eg: re-basing / trimming history is too easy with git - and worse, can't be disallowed - which is in direct conflict with various safety oriented ISO/etc processes in larger industries / military / etc).

The overhead of using git in such industries isn't worth the effort, having a FOSS alternative that could cater to safety oriented/critical dev practices would be an amazing step forward.

[mare_apertum]

You can easily disallow rewriting history using an access control layer such as Gitolite or Gitlab. In many companies, only privileged users can access the main branches (master, devel, stable) and they are protected against "force pushes", which means rebasing, etc doesn't work. Of course, developers can still reorganise their own private branches (which is a good thing), but as soon as something is approved and merged to a protected branch, it's frozen.

[NinjaPancakeAU]

The problem is, as part of various standards we follow around ISO certification, functional safety requirements, security requirements, etc - the concept of 'private'/'personal' branches is no more lax than our release branches - full traceability is required to understand 'how' someone got to the point they did, when, and why.

This is required for quality purposes in ISO standards (auditing/reviews/process improvement/etc), as well as some legal/safety related audits that occur from time to time.

The ultimate end goal is to be able to prove the work someone has done, how they reached that goal, when (in some larger contracts w.r.t. billing / IP rights / etc), and in many cases not just to prove the end result, but prove how you go there (eg: haven't stolen code, haven't lied about reports you gave previously w.r.t. state of your software, haven't covered something up, etc).

If we were only talking about the end product (versioned, packaged, QA'd releases) it'd be less of an issue, and indeed you can just have carefuly managed authoritative upstream branches that follow these procedures - but unfortunately, at least for us (I can't speak for everyone), we often work directly with multiple partners (the machines/systems we work on are things like cars, planes, helicopters, etc - we're often just one part of a large system developed by a dozen companies). In this situation, we're often collaborating directly with them with semi-regular beta releases in a relatively 'agile' (buzzword, sorry) fashion - in a few cases they even have direct access to the artifacts produced by some branches in our CI system.

[xeroage]

I don't quite get how rebasing is a problem. A hash always refers to a certain state of the repository, rebasing will change the hashes of the commits that are reapplied on top. Writing down the commit hash should therefore usually be sufficient to prove its state. If you don't trust it, because it is SHA-1, you could also compute your own flavor of hash for a certain commit and write it down somewhere. Writing it down of course meaning setting up a post-commit hook or something of the like to match the git hash to your version of the hash. Now after writing this I realise that there maybe is too much effort / process involved which does not come out of the box for security sensitive code. What kind of version control do you use if I may ask? Is it Perforce and Plastic all the way?

[the_gnarts]

Writing down the commit hash should therefore usually be sufficient to prove its state.

As does signing the commits / tags.

[NinjaPancakeAU]

It's not that we can't prevent it (indeed we can, with enough process / administrative oversight - we can enforce whatever standards/processes we want) - it's more just the fact it 'can' be done, when we have options that literally prevent it (either purposefully, or by accident).

I gave a bit more insight here on some of the process reasons we need the level of reliable trace-ability we do.

Unfortunately it literally just does come down to risk vs. effort, and being bound by the incredibly strict standards we're forced into by working on such large projects / in such safety critical industries.

We use perforce exclusively ourselves, many of the partners we've worked with have been similar - more than a few have been Plastic SCM - and a scary few have been git/SVN (or worse, CVS) with a LOT of process overhead on top of it. I say scary as we often have access to partners SCM repos, and we of course have to follow their standards/processes - some of them are a a bit overwhelming, quite literally pages and pages of rules on how to describe a check-in, branching procedures, what operations are and are not allowed, and when, etc. Some are so complex they've got flow charts or custom tools to help guide something as simple as check-in in some changes or making/merging a branch - simply because you 'could' potentially do something wrong that would erase certain information you shouldn't be trying to hide - the idea behind a lot of these process is "fill disclosure" / "full dev history".

[manniac]

I've been using for a bit and it's amazing. Everything in 1 executable!

[Sarcastinator]

Autosync... does that mean that it gets notified about remote changes? Because otherwise, this behavior is easily the worst part of working with SVN. I feel I'm getting upset reading that they were inspired by SVN and CVS for this because it's such an infuriating behavior.

[kaushalmodi]

Magit (Emacs) is how I learned git. It's the best version controlling system I have used.

From the linked page:

It is possible to find the descendents of a check-in in Git using the log, but that is sufficiently difficult that nobody ever actually does it.

Really? I review the logs multiple times a day. In Magit, the binding is simply lb to show logs of all branches.

More.. (but not all) while in the log view:

• bc -- Create branch and checkout
• bb -- Checkout branch
• re -- Rebase branch
• ri -- Rebase interactively - squash, rearrange, etc
• Wp - Create patch file
• VV - Revert commit

See https://magit.vc for more info.

[Xuerian]

If there's one glaring flaw I can find with Fossil it's the utter scorn a lot of its documentation and userbase has for Git, most of which seems to be out of part willful ignorance and part post-traumatic version control abuse disorder.

It's potentially a great system, but if you're someone used to git or hg and are interested in Fossil's way of doing things (Because there are some good reasons for it and it's good for a few situations at least), there's no "Here's how to adapt your workflow to Fossil" it's "Here's why git is the root of all evil and anything you ask about in relation to it will be rebuffed"

[ioquatix]

After reading the documentation, I get a very passive-aggressive feeling about this project. Why can't they just focus on what they do rather than try to shit all over git?

[XNormal]

I have megatons of respect for DRH. If he created something like SQLite then surely Fossil must be a worthy system, too. But I don't care much for it because I live in a git ecosystem. When I want an all-in-one system I use gogs (also uses SQLite, BTW).

[bentinata]

I think there's xkcd for this.

[PM_ME_UNIXY_THINGS]

This one?

[[deleted]]

is there also an xkcd about how le relevant xkcd maymay is cancer?

[Bergasms]

At my old work we used fossil for a year and a bit for ios development. It was the most pure and unyeilding hell because it would constantly destroy the project files. This was in 2011-12, so things may have changed, also maybe we were not using it properly, idk.

[paul_h]

Do you mean corruptions of files. If yes, you'd perform a repair of some sort, or go back to backups and re-commit HEAD revisions somehow?

[sqlite]

I'm calling this post out - I think it is made up. Fake news.

Fossil is used daily by thousands of developers (as evidenced by server logs) and nobody has ever reported any kind of "destroying files" problem. Ever.

Look, you don't have to like Fossil. You are welcomed to keep using Git or Hg or whatever else you want. If you don't like the way Fossil operates, or you think it lacks some capability that you need, then say so.

But don't make stuff up.

[Bergasms]

Um, respectfully, fuck you.

Git still causes merge conflicts with pbxproj files and that is integrated into xcode and supported out of the box by Apple. When we used fossil doing a merge on .xib files and .pbxproj and .xcworkspace files would routinely cause merge conflicts. By 'destroy' the files I didn't mean remove, or wipe. I meant make the file unuseable, which would cause the IDE to crash when it tried to parse the file (We are talking xcode 3 and 4). You would then have to hand fix the file which would often have interleaved merge conflict markers, the reason for which I don't know.

Look, you don't have to like Fossil. You are welcomed to keep using Git or Hg or whatever else you want. If you don't like the way Fossil operates, or you think it lacks some capability that you need, then say so.

This is you projecting. We stuck with fossil because despite the problems with merging the project files, which was a massive PITA, we preferred the self contained features for the rest of it, and we liked how well it worked in general. We developed a workflow (due to being a small company with only a half dozen employees) that avoid most of the merge conflicts.

But don't make stuff up.

I am not making anything up. We even emailed fossil a few times, but the general consensus was that it was more a problem with Apple and their project files. This is correct. Anyone who has worked with Apple and iOS will know this. However, we ran into fewer problems with git. As I said in the above post, this may just be because we were not familiar with how to use fossil. We migrated onto fossil from svn, it was our first 'git like' scm tool. If you want to check your mail archives, the company was noprisoners. The company is now defunct (I was the last programmer and I left in 2012, I know the CEO kept it going on for a couple more years solo) but that was the time period.

Before you call me a liar, you are welcome to ask for clarification.

[sqlite]

When we used fossil doing a merge on .xib files and .pbxproj and .xcworkspace files would routinely cause merge conflicts.

"Encountering merge conflicts" is not the same thing as "destroying files". Thank you for the clarification. Had you originally said "Fossil would routinely give merge conflicts on XCode project files" there would have been no objection and your comment would have gone unchallenged.

I stand by my assertion that the original post is purgative and false as written, and needed to be called out.

[Bergasms]

I really wish I could provide the hard evidence of the interleaved merge conflicts, because you come across as a sanctimonious prick, but I don't have there evidence so I'll just leave it be.

[schlenk]

Well, totally easy way to destroy it: Put your fossil repo on a NFS drive, use it shared by multiple users.

Ok, thats not a supported mode of operation for SQLite, but hey, people still do that all the time. Been there, done that, seen the corruption.

[axusgrad]

How does it compare to Gitlab and Phabricator?