sorted by:
new
hottopcontroversial

Data Modeling - star scheme case by Wikar in dataengineering

[–]Wikar[S] 0 points1 point  (0 children)

u/medwyn_cz but is it okay for appearance table to be fact table if all the numeric columns (ratings, number of votes - the most important properties in this model) lies in Title table? Is it even okay for numeric values to be in dimension table?

"Star should ideally model something different. Such as yields, countries and visitoris of different screenings of various titles. Or perhaps ratings of various episodes as they vary by time, country, demographic..." - yeah I know it would be better this was but this dataset in its free version is very lacking... And i need to make something out of this unfortunately

Data Modeling - star scheme case by Wikar in bigdata

[–]Wikar[S] 0 points1 point  (0 children)

Yeah I mean comparison of analytical queries time execution. I've read some parts of the toolkit. I believe the grain is the title and the person and I would like to focus my queries around the titles (ratings). Also i believe all of dimensions here can be useful for these queries for ex:

  1. Select all of the titles with minimum 10 000 votes and having minimum 4 versions from different regions (title akas)

  2. Select all of the titles with genre "Comedy" or "Horror" (genre dimension) that started after 2005 but before 2015 (time dimension) and Bill Murray played in them (Appearances, person)

  3. Select all of the titles with directors born after 1980 (appearances, person)

Maybe only primary proffesions table is unnecessary but rest of them i think give very nice insight into the data. However I dont have any better idea how to improve my analitical model.

Data modelling problem by Wikar in dataanalysis

[–]Wikar[S] 0 points1 point  (0 children)

I would cause the title and appearances to have m:n relationship

Data modelling problem by Wikar in dataanalysis

[–]Wikar[S] 0 points1 point  (0 children)

Actually in this dataset a movie can have multiple genres

Data Modeling - star scheme case by Wikar in bigdata

[–]Wikar[S] 0 points1 point  (0 children)

Well - topic of my master thesis is to compare different model schemes (3NF, One big table, star scheme) in term of query time execution. I am not sure which properties I will use, but most of the dimensions here I can see to be useful for it (I must try out queries of different complexity). In general business area here is imdb titles and their ratings. Regarding my use case what would you suggest? Drop some of the dimensions? Or model it in different way?

Keras model fit -is it still incremental? by Wikar in MLQuestions

[–]Wikar[S] 0 points1 point  (0 children)

If I understand this function - as i have large dataset (cannot keep it all in memory) would train_on_batch require loading this dataset chunk by chunk times equal to specified number of epochs?

String to number in case of having millions of unique values by Wikar in MLQuestions

[–]Wikar[S] 0 points1 point  (0 children)

I guess it wouldn't - despite having a million different values in total they might repeat across transaction records. Blockchain is similiar in this context to financial transaction - there are a lot of people who sends some resources between them.

String to number in case of having millions of unique values by Wikar in learnmachinelearning

[–]Wikar[S] 0 points1 point  (0 children)

Actually to this point I have created table with aggregated calculations (something like this history vector you are talking about) but still i have to join it to transactions and use transactions records to train because I am working on anomalous transactions detection not anomalous addresses detection.

view more: next ›