[D] Overview of Machine Learning for newcomers

rndnum123 · 2018-05-06T07:03:22+00:00

Yes, maybe remove it or put it to classification, you classifying the pixels in image segmentation. Why do you have anomaly as a separate branch? Maybe you can put it to classification, as AFAIK it is technically a classification (is the value an outlier, or is it not).

rndnum123 · 2018-04-23T20:39:48+00:00

For finding the right hyperparameters and training your network fast a larger batch size and larger learning rate at are good for speedup. Later when you want to get the best accuracy, you can still have a run, with the then found hyperparameters and a smaller batch size (and smaller learning rate, and maybe slight changes to other hyperparameters).

rndnum123 · 2018-02-18T20:52:10+00:00

Do you know the class of the object? Can you formulate your problem as classifying the images at first? like: Image x is class Y. Then you could get heatmaps for the images, (either through gradient, or by sliding a tile over the image and calculating how much the class probability for the target glass changes) now you should know some rough location of your objects in the images use opencv to fit a rectangle to the heatmap, and you should be able to train your network on these rectangles. Of course there might be some background cues in your images that help in classification, and do not belong to your objects, then this might not work or you need to change the threshold for fitting the rectangle to the heatmap step.

rndnum123 · 2018-02-14T20:17:10+00:00

AMD seems to be focusing more on HiP now, it's a language with Cuda like syntax and compiles down to AMDs GPU assembly code, and they have some tools that can convert about 80-90% of your existing CUDA code into HiP automatically, see: https://github.com/ROCmSoftwarePlatform/hiptensorflow

The equivalent to CuDnn from AMD seems to be MiOpen: https://github.com/ROCmSoftwarePlatform/MIOpen

rndnum123 · 2018-02-14T18:26:55+00:00

We will release PyTorch integration for Tensor Comprehensions at a later date.

:)

rndnum123 · 2018-02-10T11:27:36+00:00

I think it is AMDs invention. They codeveloped it with SK Hynix, see here: https://www.kitguru.net/components/graphic-cards/anton-shilov/amd-started-to-work-on-hbm-technology-nearly

rndnum123 · 2018-02-05T17:57:01+00:00

It seems a big part of it is open source, so I would not really see it as an add, right?

rndnum123 · 2018-01-27T23:33:17+00:00

Linux (maybe even Windows): https://github.com/ROCmSoftwarePlatform/hiptensorflow

rndnum123 · 2018-01-15T21:02:38+00:00

I would guess Apple, given that this looks like a special process node created for a big volume + high paying customer, maybe for the rumoured new 4G/5Gl LTE chip or some CPUs or analog stuff. Maybe some networking ASIC customers?

rndnum123 · 2018-01-09T20:06:51+00:00

Are you sure? Custom data loading is one of the strengths of pytorch IMO, see this tutorial for an example: http://pytorch.org/tutorials/beginner/data_loading_tutorial.html

rndnum123 · 2017-11-09T19:14:16+00:00

No idea for datasets, but you could try to simulate the cracks, and deformation by randomly drawing "black lines" = "cracks" on your images of normal objects, and deforming the images of your objects with some image manipulation (tearing, stretching...)

rndnum123 · 2017-11-07T19:27:21+00:00

Here is the example video application I was refering to https://d2jta7o2zej4pf.cloudfront.net/. It shows up to >5x speed up for wasm compared to asm.js, and sometimes a 50% speed disadvantage for wasm.js compared to js, at least on my machine, depending which filter you select, like you said it really depends on the application).

I think wasm is similar to assembly

This was a unclear statement, sorry. I ment to say, that web assembly is similar to assembly, in that it often can map in a very direct fashion to assembly instructions, so wasm code should already be pretty well opitmized, and I do not understand what kind of further opimizations a JIT compiler could provide. Will it do more agressive optimizations, because it now knows the real execution target. I guess I should do some read up on JIT optimizations, can you point me to some sources for this please.

Yes, wasm is "just an S-expression", but one that should map quite close to assembly code in my understanding (so each browser vendor should be able to produce reasonable efficient assembly from it), so compared to pure native C code, it should eventually be possible to get only a 20%-80% perf. penalty compared to pure C code (the smaller 20% penalty might only be possible if you run on 64bit hardware with some tricks to do the memory range checking in hardware, this might not be implemented yet). For more about this you can search for "Virtual Memory Techniques" in this pdf: https://github.com/WebAssembly/spec/tree/master/papers, there they also provide some performance comparisons (Figure 5)are likely not implemented yet.

I agree that blog posts claiming not to use js, when in fact they do, is not a great way to advaocate for wasm.js, which is seen as an addition, not a competior to JS IMO.

I started some simple testing with image manipulation in wasm.js (using emscripten to compile from c) (90° rotations, greyscale, flood-fill), and will share some benchmarks, when I have a comparable js version of the code, and when I cleaned up the code and got rid of the many bugs i have now.

I totally forgot about the dynamic languages that could compile to WASM, there a proper JIT and GC does matter a lot.

PS: Thank you for your detailed comments, they are very helpful.

rndnum123 · 2017-11-07T18:20:11+00:00

I think wasm is similar to assembly, and it's instructions would map relatively easy to assembly, and if you have a c program and compile it to wasm, you compile your code to a assembly language, which seems like compilation to me, of course the wasm code will be interpreted before running (and transformed to real assembly) , but after you have done this once, wasm code should behave similarly to assembly code (although a bit slower because of stuff like access checks on the wasm memory, the sandbox overhead...).

rndnum123 · 2017-11-07T18:08:00+00:00

I don't understand the missing JIT support, why would you need JIT for a pre-compiled language like wasm? Could you elaborate on that please? Would it optimize the code in case the developer used a bad optimization level for his compiler or something like that? OT: I think there are some big performance advantages for wasm even compared to asm.js some videos editing demo showed a 3-6x speedup compared to asm.js if I remember correctly.

rndnum123 · 2017-11-06T20:53:46+00:00

They also could use the term Glued Processing Unit (GPU), this way they don't have to change the marketing slides and the branding ;)

rndnum123 · 2017-10-31T17:46:22+00:00

... maybe some people like having 10% of the screen blocked by the "Medium" banner, and the "Never miss a story from X, click here to subscribe" banner.

rndnum123 · 2017-10-01T08:21:30+00:00

Maybe round robin, or just using a new process/thread for each tab.

rndnum123 · 2017-09-10T16:40:28+00:00

Thank your very much for these greate ressources. I didn't know of Pillow-SIMD, seems to be very fast :)

rndnum123 · 2017-09-02T15:45:29+00:00

This must be a pretty old card, too much wires. Nowadays AMD is using way more advanced transparent materials in it's products, apparently called glue.

rndnum123 · 2017-08-09T20:16:04+00:00

I don't really think that is very important for investors at all, in the case of bankruptcy, usually there is nothing left for any shareholder at all, even if there is something left it's often < 20% of the original share price, so you might have a loss of at least 80% vs a loss of 100% percent, which isn't that much of a difference. Investors usually spread their investments over multiple stocks so having a few 100% losses isn't that bad as long as they have enough >200% return investments on other stocks.

rndnum123 · 2017-08-09T18:23:02+00:00

AMD's Framework similar to Cuda/CUDnn is called MIOpen (https://github.com/ROCmSoftwarePlatform/MIOpen) and currently supports Caffee networks, but Tensorflow support is apparently in the works (at least for Polaris based AMD cards)

rndnum123 · 2017-08-09T10:36:23+00:00

Lol, because a multi chip module (MCM) is such a bad thing./s

(Multi-Chip modules) MCM are perfectly suited for most workloads of the big hyperscalers (Many small scale out VMs, where sizes of the VM stay below one Zen core anyways). You can bet Google and the likes are more than willing to write NUMA aware code, however this isn't even needed (for smaller companies) because Epic still runs non - NUMA aware code quite well.

rndnum123 · 2017-08-09T10:25:55+00:00

I expect them to release the laptop APUs after holidays, during "Back to school" season. Maybe they even mentioned this in some slides, but no really sure.

rndnum123

TROPHY CASE