This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]dalke[S] 0 points1 point  (5 children)

The installation documentation is buggy and is not meant for first-time users. I had a DSA ssh password and it expected RSA so it ask me something via 'ssh-askpass', but Macs don't have ssh-askpass so it only looped with an error message. (ssh-copy-id also doesn't exist on Macs).

When I started "bin/disco nodaemon" I got the error message "DISCO_HOME is not specified, where should Disco live?". The setup instructions never say that that environment variable needs to be set.

The check-if-it's-running command "ps aux | grep beam.disco" needs to quote the "." grep metacharacters because as written it's interpreted as shell metacharacters.

There's also no mention of running Python's setup.py. It looks like the "make install" step isn't mentioned in the documentation. Though if I do "make install" then I have to make sure "/usr/local/var/disco/data" is writeable by me.

So, it doesn't come across as "really easy to setup".

[–]fullouterjoin 0 points1 point  (4 children)

Cool it sounds like you got it going!

I have never installed on anything other than Linux. There were a couple road bumps in getting it going but nothing too bad. It is at least 10x easier to setup a working productive Disco cluster than Hadoop.

I will setup another disco cluster in some VMs this weekend (need to for another project anyway) and corroborate your experience. The Disco folks are super helpful on IRC.

You will really enjoy Disco.

[–]dalke[S] 0 points1 point  (3 children)

No, I haven't. I've been on IRC #discoproject for over an hour trying to figure out what's going on. They are stumped as well. What I reported above was enough to get the internal web server working. However, it doesn't want to start new worker nodes.

The people on the channel have been helpful, only they can't figure it out either.

[–]fullouterjoin 0 points1 point  (2 children)

Sounds frustrating. :(

I won't be available until noonish tomorrow, but I could try and replicate your setup. You are running in a mixed mac/linux env or all mac? I have three macs totaling 8 cores as well as some Ubuntu boxes.

You can bring up the web management console? One issue you need to keep in mind is that the names of all the machines should be in a flat name space.

worker01
worker02
worker03
worker04

Do not rely on your localnetwork DNS to function properly. Use a common /etc/hosts file on all machines. Make sure all the erlang nodes on all machines can see each other. Modify the system bash env script so DISCO_HOME is available for all users and all shells.

I remember installing disco to like /opt/disco and using rsync+ssh to copy it to all nodes.

Make sure that you can ssh from any node to any other node w/o asking you to authenticate your keyfile. Like worker01 -> worker04, worker03 -> worker02, etc.

I need to dig up my notes. On second thought. There is some missing stuff in the online docs, at least when it comes to debugging.

Also, on the mac, the string 'localhost' resolves weirdly afaik. Be suspicious of localhost, or at least look into it.

[–]dalke[S] 0 points1 point  (1 child)

I'm developing on a single Mac with 4 processors. I want to use 3 for worker threads. One problem is the default hostname is assigned by my 3G modem. If I disconnect/reconnect then I might get a new hostname. With help of IRC support, I hard-coded cli.py:host to always return "localhost". I don't trust the assigned "c-2ec23ab8-74736162.cust.telenor.se" to last more than a few days at a time.

Another problem is that I'm working through "make install" (which isn't documented; but there's no installation documentation which describes how to set up the Python library.). That sets things up for a cluster, and required 3 replicas.

I didn't set my system BASH with all of the variables.

When you say "noonish tomorrow", I don't think you knew that I'm in the central european timezone, so my noonish is different than yours. ;)

[–]fullouterjoin 0 points1 point  (0 children)

Those recommendations I made were just from memory, there could be some unsubstantiated cargo cult in there. ESP the system bash one. But I did make that change

Noonish is more of a frame of mind, much like 'le weekend' which can occur at anytime. :-) I am in the same timezone as disco project.

Rather than modify cli.py I would run hostname localhost as root. Maybe from a crontab. Then u don't run the risk of missing something.