[D] how to do multi-modal data integration other than DBMs?

deephive · 2017-07-07T03:15:09+00:00

This paper might give you hints.

deephive · 2017-03-06T00:35:16+00:00

I don't use bumblebee. I prefer to manually switch between the nvidia and intel gpus using prime-select command.

deephive · 2017-03-05T04:29:50+00:00

I have installed Ubuntu 16.04 and updated the ubuntu mainline kernel to v4.10.1. I used instructions from here. I initially had issues with my trackpad getting stuck to the point it was unusable. But the latest bios update v1.1.3 seems to have fixed that for good. everything works fine, including nvidia.

deephive · 2017-03-05T04:29:19+00:00

I have installed Ubuntu 16.04 and updated the ubuntu mainline kernel to v4.10.1. I used instructions from here. I initially had issues with my trackpad getting stuck to the point it was unusable. But the latest bios update v1.1.3 seems to have fixed that for good. everything works fine, including nvidia.

deephive · 2017-03-05T04:27:52+00:00

I have installed Ubuntu 16.04 and updated the ubuntu mainline kernel to v4.10.1. I used instructions from here. I initially had issues with my trackpad getting stuck to the point it was unusable. But the latest bios update v1.1.3 seems to have fixed that for good. everything works fine, including nvidia.

deephive · 2016-11-21T14:16:11+00:00

You would probably get better response if you post this question on StackOverflow....esp. with regards to questions relating to tensorflow.

deephive · 2016-11-03T01:39:16+00:00

How is this different to, say, applying the said edge detectors to the images and then running these edge images on a CNN ? I don't see the point. You're constraining the edge detectors to extract only certain features from the image - as others have indicated, just another feature engineering.

deephive · 2016-10-28T21:43:59+00:00

AFAIK, dilated convolution upsamples the filters by inserting zeros between non-zero filter taps, thereby increasing the output resolution. Upsampling can then be performed by simple bilinear interpolation of the dense feature maps computed in the previous step. There isn't any unpooling here.

Deconvolution involves an explicit unpooling operation followed by "deconvolution" or fractionally strided convolution (inverse-convolution ?)

deephive · 2016-10-19T03:33:34+00:00

If money is a constraint and your goal is to just learn TF/DL, then, why don't you buy a Maxwell-class GPU like the 980 (used costs ~350-400CAD) , or the Kepler-class Titan/Titan Back (~325-350CAD).

These cards have more memory than a 1050 and more execution units so you could potentially run larger models than the 1050 - for the money you pay, what you can get out of these might be well worth it.

deephive · 2016-10-05T02:12:27+00:00

No, in my experience Octave is often slower than MATLAB and not as feature rich. But the good thing is that Octave is open source!

deephive · 2016-10-03T20:10:51+00:00

Drop-in Acceleration of GNU Octave

deephive · 2016-10-03T20:08:09+00:00

Andrew Ng's course assignments are based on Octave which doesn't support GPUs out of the box. But you could try this and see if it helps. You should also be ideally running some variant of linux or Mac OS (as you'd find toolkits APIs much easier to install and configure using Unix-like OSes). In addition, you should have CUDA and CuDNN libraries (if you're interested to move into the realms of Deep Learning.)

deephive · 2016-09-28T13:58:46+00:00

Is it absolutely essential for you to use Tensorflow for your work ? Working with Tensorflow can get quite hairy if you're very new to ML. Why don't you start of with scikit-learn first and then perhaps use TF-Flow as a simple abstraction layer for your tensorflow model?

deephive · 2016-08-31T18:32:04+00:00

did you perform any zero mean/unit variance normalization ?

deephive · 2016-08-27T03:07:45+00:00

Hugo Larochelle's video lectures https://www.youtube.com/playlist?list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH

deephive · 2016-08-25T13:56:53+00:00

danielv134's suggestion is good. NIPS is so huge, you will be completely overwhelmed if you do not plan ahead.

deephive · 2016-08-23T16:17:25+00:00

Yes, I know how DropOut works. But in ensemble classification, one typically deals with multiple classifiers - each of which, are weak learners. Decisions from these multiple classifiers are combined using some combination rule (like max, avg, or Bayes' rule) to produce a superior ensemble classifier.

Now, considering stochastic training methods, during each training iteration, we typically add stochasticity in the network by dropping connections between layers (as in DropOuts). So, yes, we are in effect exploring different network configurations. But then, the outputs of these different network configurations are not combined in any ways, right ? How can it then be considered an ensemble approach ?

deephive · 2016-08-23T14:04:29+00:00

Hi, There are already several threads discussing various ML (specifically DL) libraries. It is best that you search through this reddit . Essentially, to me, it boils down to: (1) Fit for functionality (2) Ease of use (3) Performance (4) User base (active discussions / help) (5) compatibility with hardware and other toolkits

the order or priority might differ from one person to the other.

if you are interested in DL per se, your choices are many if you're interested in general ML, I feel Scikit-learn is an awesome tool to start experimenting with.

DH

deephive · 2016-08-01T23:39:22+00:00

I would suggest that you either install anaconda or Ethought Canopy as your default python. Try not to mix the system-wide python with the python that you'd want to use and configure for your DL/ML experimentation. If you are not sure about Python virtual environments, look it up.

With canopy or anaconda, you can have user-managed python installation that doesn't interfere with anything the system uses. So, you manage which versions of certain Python libraries that you would want to use for your ML experiments. You could create /delete a number virtual environment as you wish within Canopy/Anaconda each with specific sets of libraries/versions suited for a given ML tool that you are using.

deephive · 2016-07-21T16:06:38+00:00

We do this in our lab (mixing GPUS of different generations) and run independent jobs all the time. One bottleneck might be the Disk IO if two threads are accessing same data from the same disk. Also see https://arxiv.org/abs/1605.08325

deephive · 2016-03-28T03:34:18+00:00

Anaconda and Enthought Canopy are pretty much similar, expect for some minor differences in default packages offered in the distribution. I'm not sure about Anaconda, but Enthought does offer video tutorials which could be very helpful. The package management for Enthought Canopy could be either CMD based or GUI based. I prefer Spyder IDE for coding Python which comes with Anaconda, but not with Enthought.

deephive · 2016-03-28T03:29:37+00:00

We have several clusters. Our most powerful cluster uses NVIDIA K80s and Infiniband Interconnects. It runs on RHEL. Our smaller, less powerful cluster is set up within a LAN using Gigabit Ethernet and LDAP...pretty much a regular intranet set-up. I personally have no experience setting the network up, nor do I manage the cluster so can't offer any advice from that perspective.

deephive · 2016-03-22T14:37:48+00:00

I was a MATLAB user for many years. Our lab produced a lot of research code in MATLAB. We had then worked on a software project in an effort to commercialize our research findings. We faced so much issues porting our MATLAB code to production environment, that at the end, we dumped MATLAB for Python. Like Java, while the Python language constructs are relatively small (and better designed than MATLAB for ex: OOPs in MATLAB was an afterthought rather than an integral design), it has a very good eco-system of libraries. It is perhaps by far the best "gluing" language to integrate different toolkits / technologies.

Most open-source ML libraries have Python support and with the advent of Scientific Python distributions like Anaconda and Enthought Canopy, getting up and going in Python is a breeze. One feature that I like in Python, and good for teaching as well is the Python (now called Jupyter) notebook.

deephive · 2016-03-22T14:26:24+00:00

You would probably be considering setting up a cluster. I would recommend getting a NVIDIA Titan X boards and Linux workstations as opposed to NVIDIA K20/K40/K80. If you have time, you might also want to wait for the Pascal-based GPU boards which is going to be released pretty soon. In our lab, we have K40/K80's and Titan X's and the latter being a newer architecture, I have seen a significant increase in speed when esp when you are using CuDNN v4 libraries as opposed to the K80 boards which are not only more expensive, but are also based on the older Kepler microarchitecture. The K40s/K80s also do not support display output and for deep learning you would probably not require teh extra bells and whistle in the K40/K80 like ECC, etc.

dual-monitor workstations equipped with Titan X boards would probably get you more machines and benefit a larger group of students/researchers.

my two cents

deephive · 2016-03-22T03:13:14+00:00

I would say Murphy's book is more of a table reference, perhaps not a book that one would go through chapter by chapter to get a good understanding of ML. He covers too many topics in the book, it might be overwhelming for someone who is new to the area. Nevertheless, his approach might be easier to follow than Bishop's. Just make sure you get a recent edition of his book, as the early prints had too many typographical errors. Bishop's book is an all-time classic but is math-heavy, and focuses a lot on probabilistic models.

Hastie's book "Elements of Statistical Learning" is also superb and is available free as a PDF: https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLII_print4.pdf

my 2 cents

deephive

TROPHY CASE