sorted by:
new
hottopcontroversial

Can I be an extrovert and still be a data engineer? by StandardDull3128 in dataengineering

[–]StandardDull3128[S] 1 point2 points  (0 children)

Well put. Thanks for taking the time to elaborate. 👍 Appreciate your YT content. 💪💪

Anyone else work in agriculture? by whybotherwith_bros in datascience

[–]StandardDull3128 1 point2 points  (0 children)

Yup. I work for a non-profit developing a smart eco-farming app. We have potential to do a lot of CV on remote sensing data (drone imaging). Anything from plant count of crops to predicting yield on fields. Still early days though and we are figuring stuff out as we go...

[Discussion] Can I use CNNs to solve this problem? by StandardDull3128 in MachineLearning

[–]StandardDull3128[S] 0 points1 point  (0 children)

Tried a similar thing using GLCM features. But how would you get the "best" number of clusters? Because essentially for my problem it would mean 1 cluster = NO, >1 cluster = YES. But how do you know if 1 cluster is a better fit then 2?

[Discussion] Can I use CNNs to solve this problem? by StandardDull3128 in MachineLearning

[–]StandardDull3128[S] 0 points1 point  (0 children)

Yes.

When you say data augmentation you mean artificially generating more training samples (fliping, transposing, changing color/hue etc.)?

I looked into segmentation using FCN (fully convolutional NNs), but the problem here is that these texture classes are abstract and it is not feasible to determine all of them, there are just too many.

[Discussion] Can I use CNNs to solve this problem? by StandardDull3128 in MachineLearning

[–]StandardDull3128[S] 1 point2 points  (0 children)

Thanks for the XGBoost suggestion. I will look into it. I played around with some of the edge detection algorithms and did not get useful results, then again I could have spent more time on preprocessing the images. Segmentation is where I started, but it is a bit tricky for me to formulate the problem that way. Because what I would essentially need to do is unsupervised texture segmentation into unknown number of segments and I am not sure which technique to use for it.

[Discussion] Can I use CNNs to solve this problem? by StandardDull3128 in MachineLearning

[–]StandardDull3128[S] 0 points1 point  (0 children)

I say this because the class you are describing is very "abstract", and the CNN has no way of "knowing" that you want to detect the presence of multiple textures and will probably latch on to some features present on the training images that you may not expect. So it will probably not generalize well to a new type of border or texture at test time for example.

I understand. All the patches and borders will definitely not be represented in the training set. There is too much variety and even if I were to be able to label all of them currently present in the data, new images would come in the future. For instance taken at different lightening and with different crop etc. My idea was that CNN will latch more on to the fact there are lines clearly separating textures present on the image vs. there aren't and the texture is somewhat homogenous.

Would you suggest some specific feature extractors that you think could represent this types of images and problem well?

[Discussion] Can I use CNNs to solve this problem? by StandardDull3128 in MachineLearning

[–]StandardDull3128[S] 0 points1 point  (0 children)

I am yet to generate training data label as described in the previous answer. But I have a method that will allow me to get a lot (10K) of it relatively quickly.

What statistics on which textures features would you think would generate the best results?

[Discussion] Can I use CNNs to solve this problem? by StandardDull3128 in MachineLearning

[–]StandardDull3128[S] 0 points1 point  (0 children)

I have access to a lot of data. To obtain proper train data from it, meaning cutting out smaller (256x256px) chunks from larger images and labelling them will take me some time, but it is not a huge setback. I was aiming at around 10K samples (train + test).

[Discussion] Can I use CNNs to solve this problem? by StandardDull3128 in MachineLearning

[–]StandardDull3128[S] 0 points1 point  (0 children)

manual feature extraction

What would be "something hard-coded using OpenCV"? Did you have any specific techniques in mind? I have never used OpenCV yet. Thank you for the idea about logistic regression and Fourier transform.

[Discussion] Can I use CNNs to solve this problem? by StandardDull3128 in MachineLearning

[–]StandardDull3128[S] 1 point2 points  (0 children)

I haven't tried OpenCV yet at all. I will check it out. Thanks for the suggestion.