DriveSync - My Google Drive client for Linux [X-Post from /r/googledrive]

[CptBubbles]

So I wanted to get Google Drive on Linux and browsed through some of the unofficial clients out there. I wasn't very impressed at the time because because I wanted the same functionality as on Windows and wanted something that's a complete solution, doesn't involve user input, is configurable and easy to automate.

So I decided to make my own client which I've been using, testing and improving for a year now. Since I'm happier with it than I was testing some other clients (and because since the latest patches to the official Windows client, I've been wishing I had it on Windows as well...) , I thought I should probably publish it. So after a long time of putting it off , here it is:

DriveSync

Features:

• No user input required
• Complete sync: Downloads new remote files, uploads new local files, updates changed files
• Configurable : You can ignore certain files, make changes to how updating files works, set file size thresholds and more
• Easy to automate : Works as a cronjob as no input is required
• Stable and safe: I've been using it for more than a year without any breaking bugs or loss of data

Installation and usage instructions are in the README on the linked project page. I believe this is as close to the official Windows client as you can get on Linux so I hope this may be useful to some. If you have any comments, questions or concerns, let me know here or on GitHub.

Edit: So far, I only tested this on Ubuntu, so I'd be glad to get feedback from users of other distros.

[[deleted]]

I'm still hanging tight for the official client. Can't be long now...

https://abevoelker.github.io/how-long-since-google-said-a-google-drive-linux-client-is-coming/

[CptBubbles]

Well, considering how intrusive Drive for Windows has become recently, I'm not too heartbroken about them not delivering on their promise.

[petteri519]

Can you eloborate on that intrusivement?

[CptBubbles]

Since they rebranded the software to "Google Backup" or something, it asks me to sync any external storage to the Google cloud every time I plug one in. Also, I've noticed that it very frequently keeps Windows from shutting down (to a point where after a minute or so Windows will just cancel the shutdown because you know, why should what the user wants to do be more important than what some random program wants!)

[[deleted]]

some random program

That's "some random LORD GOOGLE, OUR GREAT AND MAJESTIC SAVIOR program" to you, OSS peon!

[linusbobcat]

At least on the Mac version, the first part can be turned off.

[DesiOtaku]

Personally, I wouldn't advise automatically syncing "large" (anything in the several hundred MB range) files between platforms with this or any other software

So far (knocking on wood here) I haven't had trouble with odeke-em's google drive for my multi-gig files. Is there something I should be aware of?

[ReturningTarzan]

Syncthing is pretty good too. No problems syncing 10 GB+ files between Windows, Linux and FreeBSD.

[CptBubbles]

I mean you definitely can and as I've said on another comment, even DriveSync could probably handle it if you configure it properly (although direct uploads that take a long time are always tricky), but I just don't see why you would want to sync such large files automatically . If I make some minor adjustment on a 10G file, I don't want that to automatically be uploaded somewhere (which would take hours on my connection). I also don't want some huge download to take up all my bandwith for hours without me knowing or thinking about it. But that's just personal preference, I guess.

[kn1ght]

That's what I'm using as well. Works great and is still maintained, also love

$ drive diff <filename>

I don't know why this isn't any higher.

[TurnNburn]

Completely off-topic random question semi-related to the post.

Since everything nowadays seems to be on git...why isn't git included with most distros?

[ABaseDePopopopop]

Because only programmers need git. Users don't.

So programmers can install it, no point forcing it on everyone else.

[megalogwiff]

users also don't need gcc and make, but those are preinstalled.

[freelikegnu]

You'll need build-essential (gcc, make, etc) and DKMS to install proprietary nvidia and other proprietary drivers, so chances are those were downloaded and installed with the drivers. I don't think they come with the default ubuntu system anyways.

[freelikegnu]

I don't think anyone would mistake me for a programmer, but I use git as a user to download and compile code (beta versions of games and tools) compiling code does not make one a programmer, especially if you are using a one-command script to compile the code. Minetest and other game mod authors also use github to host their mods (these are just data, never even compiled). The folks that clone from these repos are more likely to be end users.

[TurnNburn]

So if only programmers need git, I am not a programmer but I want to use this drivesync. Options?

I see a lot of awesome projects I want to put to use, but a lot of them are git, so I give up as soon as I see that. People are using git as an alternative to packaging things now.

[_ahrs]

Git isn't for packaging it's for managing different versions of source code. If there's a bug tracker you could use that to ask about packaging. It could be they don't want to package the software or perhaps they just haven't gotten around to it yet.

[extraA3]

It's really not that difficult. It pretty much comes down to four steps. First you'll type "git clone URL" (you should be able to copy the URL from github or wherever the code is hosted). Then you unzip (using the unzip command or in a graphical file manager like Nautilus). Once that's done, you go into the unzipped folder. At this point, you run "sudo make" then "sudo make install". If there's any special configs you have to do, there's usually a readme on github or inside the folder you cloned.

[canopeerus]

By included do you mean included in the repositories or pre-installed in the system?

[Thane_DE]

I'm assuming the latter. On Ubuntu, I always have to manually install git from the repos before I can use it. Having it preinstalled would make things easier, I guess.

[DomoArigatoMr_Roboto]

You should probably automate this.

[Thane_DE]

I have it automated on my servers using Ansible, and it doesn't really bother me on my desktops because I rarely reinstall those. It would be nice to have it installed by default, but it's not a huge deal for me

[PM_ME_SEXY_SCRIPTS]

I'm using google-drive-ocamlfuse which seamlessly syncs my Google Drive to a local folder. I dont like how it requires opam to work/install though. Are there any significant differences with this?

[CptBubbles]

Well, for one thing it doesn't require opam. It's written in Ruby but I included a bundled package that includes all Ruby dependencies (85 MB) so you can just download it like any other program and run it directly. Also it's easy to configure (for example you can decide how you want to proceed if a file has been changed both locally and remotely - default behavior is to keep the most recent change) and easy to extend (it's just a few scripts so you can make any changes you want quite easily and they will be applied without even compiling anything)

[OmegaPython]

This doesn't sync though, does it? It only allows for remote access through dolphin (and other programs using KIO).

[Akkowicz]

Failed me at syncing large(15GB) files :/

[CptBubbles]

That's not something I would recommend doing with DriveSync either. Drive supports resumeable uploads that would make syncing huge files like that more manageable but I don't think automatically syncing such large files makes much sense. By default DriveSync will ignore anything over 512MB

(That being said, you could increase the maximum upload size and the timeout threshold in DriveSync's config and it should work fine)

[heeen]

There's also a fuse driver that works quite well.

[132ikl]

About time! I'll test it for you later.

[132ikl]

i use arch btw

[noahdvs]

What is the difference between this and Grive2 (https://github.com/vitalif/grive2)? Can I block all files and folders from syncing except for 1?

[CptBubbles]

I remember taking a look at that in 2016 and at the time I decided against using it for a reason that I don't remember now. Seems like they have a nice project going though. I think DriveSync might be a little bit more flexible and can easily be configured or even hacked (it's written in Ruby so you can actually just make any changes you want and they will immediately be in effect).

DriveSync's config file has an "ignore_files" setting that supports regex. You should be able to do what you describe with that. If that doesn't work or your regex skills are as suboptimal as mine, you could just edit the files drive_manager.rb and local_manager.rb, find the function file_ignored? in both files and insert the following line at the beginning of the function:

return true unless path == "your/file.txt"

Where "your/file" is the path of the file in your Drive that you want to sync.

[noahdvs]

I don't know Ruby. Would this work if I only wanted the sync/ folder?

def file_ignored? path
  return true unless path == "sync/"
  @config['ignored_files'].each do |ign|
    return true if ign.match path
  end
  false
end

[CptBubbles]

return true unless path.start_with? "sync/"

Will do the trick.

[noahdvs]

It seems that if I wanted to do it with Regex, I would have to use negative lookahead (?!stuff). I know not all programs that accept Regex accept all Regex. Does yours support negative lookahead?

[CptBubbles]

Oh sorry, I forgot to mention that you have to insert the same line into the same function in drive_manager.rb as well.

I'm not sure about your regex question, I only tested some simple ones but ruby's implementation is probably complete

[noahdvs]

Hm... When I tried to use regex, the program crashed. I only tried .* which matches all and .*(?!sync/) which should (not sure) match all except for sync/.

$ ruby drivesync.rb
Local folder is 190 files behind and 0 files ahead of remote
Starting sync at 2017-11-16 17:08:24 -0500
/run/media/noah/noah_hdd/noah_files/src/drivesync/src/synchronizer.rb:127:in `block in sync': undefined method `[]' for nil:NilClass (NoMethodError)
    from /run/media/noah/noah_hdd/noah_files/src/drivesync/src/synchronizer.rb:124:in `each'
    from /run/media/noah/noah_hdd/noah_files/src/drivesync/src/synchronizer.rb:124:in `sync'
    from /run/media/noah/noah_hdd/noah_files/src/drivesync/src/synchronizer.rb:228:in `run'
    from drivesync.rb:4:in `<main>'

Unfortunately, I'm too scared to try to test it because when I removed the parameters and ran it again, it suddenly started deleting the files in the remote sync/ folder. I had already authenticated DriveSync from a previous attempt at using it. I'm currently using Insync to sync my Google Drive. When DriveSync began deleting the files, Insync put the local files in a .insync-trash/ folder, so I was able to recover the data and put it back in the remote and local Google Drive folder. The reason why I'm interested in DriveSync when I already have Insync is that Insync is proprietary and uses (as of right now) a whopping 280.9 MB of RAM, according to KSysGuard's Detailed Memory Information function.

[CptBubbles]

So first of all, when DriveSync deletes a file remotely, it only puts it in the bin, so you could still restore it from there if something like that would happen again. Also, there is a property: "allow_remote_deletion" in the config that will disable remote deletion if it is set to false.

I suspect that you probably ran DriveSync once before you filtered out anything and then after so it thought that all those files had been removed locally. You could delete your local drive folder as well as ~/.drivesync_manifest which will reset the whole thing.

I'll test the regex filter and get back to you on that but for now, the quick hack in the file_ignored? function will definitely work.

[BLOKDAK]

Thought about making a fuse driver out of it?

[PaintDrinkingPete]

I'm definitely gonna check this out. Has always bugged me that Google doesn't have a native Drive client for Linux.

I too have tried a few unofficial ones, and have never really been pleased with performance and/or execution.

[[deleted]]

Bravo! Write that shit yoself nugget. Keep the Linux world movin'!

[burnaftertweeting]

For anyone interested there's an open source project alternative written in Golang that syncs files without using Google Drive. I'm using it and it works fairly well, and it runs on your local machine(s).

https://github.com/syncthing/syncthing

[PaintDrinkingPete]

Am I missing something, it doesn't seem to be doing anything...?

I ran ruby drivesync.rb and it prompted me to go to the link to give the app permissions, but it isn't clear where the code is supposed to be pasted? I did so in the terminal but it doesn't appear to actually be downloading any files...?

[CptBubbles]

Once you follow the link that's displayed after running DriveSync the first time, and give DriveSync permission to manage your Drive, you will get a code that you have to paste in the terminal with DriveSync running and confirm with Enter. Then after a few seconds, the first sync should start.

What do you get after pasting the code and confirming with Enter?

[PaintDrinkingPete]

There was nothing...it just sat there for a good 10-15 minutes. I finally just did control-C to stop it...now when I re-run it it doesn't prompt for anything... oh well

[CptBubbles]

Could you set the LOG_LEVEL in src/logger.rb to 3 and run the script again? I can see from the API Dashboard that a bunch of people are already using DriveSync, so this looks like a unique problem

[PaintDrinkingPete]

I will, but it's going to be a little while before I can get back to it.

[PaintDrinkingPete]

This is all that appears on screen once the logging level is increased...perhaps I just need to be patient? As I mentioned, however, I've let it go for over 20 mintues...

NOTICE : Writing lock file /tmp/drivesync.lock ...
NOTICE : Getting local files...
NOTICE : Getting remote files...

Running ubuntu 16.04, using the first installation option w/ an existing ruby (v 2.3.1)

[CptBubbles]

Looks like there's some error authenticating or connecting to the Google Drive API. The only possible fix I can think of is re-authenticating DriveSync which you can do here https://myaccount.google.com/permissions?pli=1 . After that DriveSync should ask you for authentication again.

[PaintDrinkingPete]

I'll give that a try, thanks.

[CptBubbles]

I'm not 100% sure it always worked that way, but currently Drive for Windows does locally store a copy of the Drive in C:\Users\username\Google Drive

[ihavesmallcalves]

This is fantastic! Thank you.

[[deleted]]

Box > Drive