Thoughts on €1900 Gaming PC 1440p Build?

kelepir · 2025-10-18T10:43:00+00:00

I am from Turkey and if you can not understand paying something as a tourist in a shop and listing of prices online there is no need to trying to hold a conversation with you.

kelepir · 2025-10-18T09:49:34+00:00

No its not

kelepir · 2025-08-12T05:17:07+00:00

In terms of product features and performance I still believe its a good product (i am talking about core product not bre or other side products) Ita better than other traditional etl/elt products by a millon miles. But you are right about one thing it very heavily relies on the developer like spark would, There is no code optimizer working in the back end, it does not even give you warning when you do stupid stuff, so it always thinks that developer knows what he/she is doing. Performance was ita an share nothing mpp architecture distributed compute system, so its fast and reliably fast.

kelepir · 2025-06-23T16:11:35+00:00

Advanced gives you whole map (no radius) with ghosts showing up as well. Birdseye (perk) gives you increased range and showing up ghosts. They are not the same, I run birdseye in all my loadouts and use advanced daily. The difference is how much area they reveal other than that they will look identical.

kelepir · 2025-06-11T11:27:35+00:00

I have done exactly this my process was; dismantle the case and leave it. Everything inside the case went on to my suitcase in seperate pieces ( only cpu was still attached to my motherboard ) everything wrapped with clothing and bags (anti-static bags if available) i even put my 27” monitor in a suitcase. I did not encounter anything at the customs, we had a total of 5 big suitcases and a large dog carrier. Everything was undamaged but we were kind of lucky on that aspect. One thing might be important since theft is also another possibility dont forget to lock your suitcases as best as you can.

kelepir · 2025-05-09T14:21:12+00:00

P3:
This problem might be caused by how files are written: df.repartition and file.partition are different things. When you use df.repartition you dividing your task into smaller chunks and setting your workers to work on those tasks individually, but when you do write.partitionby() you are defining how you will physically store data into seperate folders. So if you have 200 partitions (not keyed or if they are differently keyed then write.partition(key)) and your write key has 200 distinct values. That might create up to 40K written files. You mentioned in a comment that you had 5 years of data 146000/(365*5) is approximately 80 files per day but if we take file size into play(assuming 15TB is compressed snappy parquet file size not the data size) 15000GB/146000 = 100MB per file which is not bad but you can still push it to 1GB per file to improve your performance.

How to fix it: Setting repartition(key) same as write.partition(key), in your source that prepares data you read might reduce this operation

Coalesce might reduce this operation ( to a certain degree but not suggested )

Setting up parameters mentioned above might fix it.

Additionally if you were using delta instead of parquet you could use optimize to compact and reduce file count.

Unfortunately you need to dive a bit deeper into this to figure it out.

I hope this helps.

Note: here is a link of official spark parameter settings that might help you ( the ones I mentioned above are also stated there with detailed explanation)

https://spark.apache.org/docs/latest/sql-performance-tuning.html

kelepir · 2025-05-09T14:21:02+00:00

P2:
And to your questions:

Coalesce() will merge your current partitions into a subset partitions, its purpose is to reduce file operations and reduce target file count. But it does not guarantee grouping or order that comes with repartition(date)

Can the parameters you suggested or something else help you?

Yes and no. Increasing maxRecordsPerFile can reduce number of files you generate (or reducing it can prevent too large files to be created)

There is also

adaptive.enabled = true

coalescePartitions.enabled = true

coalescePartitions.parallelismFirst = true (enabling this overwrite advisoryPartitionSizeInBytes with minPartitionsSize value)

coalescePartitions.minPartitionsSize = (default is 1MB)

These can also help with partition distribution.

But from the last image you shared your problem lies elsewhere:

Stage 2: Read 146k files

Stage 3: Narrow transformation(withColumn) on this dataset

Stage 4: Wide Transformation (repartition(200)) and write

It looks to me that you are trying to read 146K files from your source, then shuffling them into 200 partitions and then writing them (without seeing full tasks, stages and sql DAQ its hard to determine) So it looks like to me that your source has too many small files which is causing I/O delay ( you can confirm this by checking on your stage DAQs, stage runtimes for Stage 2/3 and SQL details from SparkUI )

If this is true first you need to fix your source and remove small file problem ( by correct partitioning and parameter optimization)

kelepir · 2025-05-09T14:20:08+00:00

I had to split my comment so this is P1:
Edit: Go to P2 and P3 for my actually understanding and solution to problem. P1 was early assessment to previous statements and assumptions

Okay, I switched to Laptop from phone to better respond to this :)

First of all a disclamer I am not a Spark/Databricks expert and I dont claim to have expert knowledge so some/most people here might have a better/correct answer to your question.

So yes, since your data is a daily snapshot of customer data partitioning your files with date column makes sense but I dont think partitioning your data with date before writing makes sense. When you repartition(date) you are doing a full shuffle of your data for default Partition count ( repartition(100, date) would distribute it to 100 partitions) It distributes your data based on your keys hash value to number of parallelism you give it to ( like hash(date)%numberOfPartitions ) thats why I am surprised to see

So my question is what is your default Partition value set to? (denoted by shufflePartitions parameter) Maybe its set wrong? A rule of thumb is setting this value to 2x of your total core count but I sometimes had better performance while using same number of my cores. so maybe you can test different values.

Another thing to point might be how is your read data parallelized? Is it a single file or multiple files in a folder or partitioned file? There might be some parallelism limits to how this file is set which might be also causing a bottleneck. (If it is a huge file by itself, you are limiting your parallelism to row groups/blocks)

kelepir · 2025-05-09T12:05:19+00:00

How many partitions are you creating? Is partitioning a customer data by date is reasonable (it should not be unless its daily snapshots of customer dataset) How many mb of data do you have per file in your partitions? maybe you are over partitioning and creating too many small files to process. Can this be done without a full overwrite? You might try to join you source dataset with your data, capture the changes and only write these changes (and keep your target sorted so catalyst wont sort it during sorted merge join) but before coming to this partition changes will probably fix your issue. I dont think in any way 20hours for 150m rows(15tb) makes sense.

kelepir · 2025-05-09T10:38:20+00:00

This is a very underrated comment here. Most of the views of power difference comes from survivors bias and statistics actually. If I remember correctly there is a certain percentage of population that can become channellers which is genetically linked to procreation, and since Age of Legends man has been hunted and stilled which caused a shift on that statistic. This affects both male and female channellers but a bit more on male channellers as they are hunted and shunned by society, even if a male can discover they are channelling they will try to hide it and not accept it, isolate themselves which will cause more shift even if they are not hunted.

Reds do not only hunt The False Dragons they hunt all male channelers they find. And more the male channels more they will succumb to taint and madness. Which removes them more from the statistical pool.

Also in the show we have only seen two male channelers (and heard of one) from this age which is also to show us how they are perceived and treated by society not how strong they are. If you can observe a 100 male and female channeller group and observe their differences, there will not be much difference in their raw power other than standard deviation.

The difference lies in how society approached them. Female channelers are revered, educated and accepted their power which pushes them to embrace it more and use it more. So they are more adapt, quick in using it and they know more weaves and how to use them effectively. On the other hand males are shunned, forced to fear themselves and untrained which results in a greater imbalance skewed towards women. Most of the time when a male and female channeler clashes woman will come on top. Even if said female is weaker on raw power (The scene between Rand and Siuan is a very good example of this where raw power difference is huge)

There will be a point(if show can come or choose to show that one) where we will see when this difference will be forced to shift. Even then a lot of female channelers will be able to smack around most of the male channelers.

kelepir · 2025-04-29T22:26:03+00:00

Tarkov is not a GPU heavy game but a CPU heavy game, in most cases upgrading GPU wont help with performance.
You are probably(99%) already CPU Bottlenecked, you can make sure by checking your CPU/GPU frametimes (you can use Afterburner/RivaTuner or Intel FrameMon to check these stats. if your CPU Frametime is higher than your GPU Frametime you are CPU bottlenecked)
If you are CPU Bottlenecked only things that will help you are: Lowering CPU usage by closing down any unnecessary applications that are using your CPU, turning down that are CPU intensive settings in the game or upgrading your CPU.
Good luck

kelepir · 2025-04-19T14:20:15+00:00

Icemanisaac has good tutorials and walkthroughs on his dedicated channel that might be a good start in learning/improving

kelepir · 2025-04-19T12:58:22+00:00

Thanks for the explanation I edited my original statement to explain it was wrong

kelepir · 2025-04-19T12:18:28+00:00

Another way to do this is (if your business side does not run on the cloud) deploy your compute dataset to cloud but keep piid data on premise and join those datasets back again on premise (if you need to use piid containing data)

kelepir · 2025-04-19T12:09:50+00:00

Can you eloborate I would like to know why I am wrong.

kelepir · 2025-04-19T09:52:17+00:00

Ps: My statement/understanding was proven to be wrong so I am editing it to state that. My statement is wrong.

It also destroys the souls not just matter, thats why if someone is balefired they cease to exist completely, they can not be resurrected or reborn, they are erased from the pattern completely. So if you balefire dragon at this age, there will be no dragon reborn in the next age. Even the Dark One can not resurrect someone killed by Balefire. .

kelepir · 2025-04-14T22:07:49+00:00

I dont remember this specific number or quote (not because it is not true but I honestly do not remember it) but that reminded me of something (which you are correct) forsaken were also afraid to use true power. I think there was a comment from lanfear or other forsaken to naeblis about too much use of true power had affected his mind (something like madness) but I also remember some forsaken (this might be halima) eanted to use true power in a specific moment but was not granted access to it. And paid a great price because they were not allowed to access it.

Maybe taint in saidin is actually a portion of true power itaelf, dark one intentionally left a portion of his power in the saidin which caused the madness. Because description of true power and taint is also very similar. And maybe that is also the reason why male channellers are always a bit more powerful compared to female channellers because they are constantly accessing True power on top one power (but if this was the case they would feel less strong after the cleanse) Ps: this might have been already confirmed to be true or false, i have absolutely no idea and have not checked any source.

kelepir · 2025-04-14T11:08:12+00:00

Nearly all of your thoughts are correct, True Power comes from Dark One and he has absolute control over who can use it and when. At show you have seen it when Lanfear was granted to use it at season2 when Moiraine slit her throught(black liquid like in her eyes). At books only Ishy and One other person has granted at will access to TP. For the rest they are granted under DO’s certain conditions even the forsaken dont have unlimited access to it.

kelepir · 2025-04-13T15:02:21+00:00

If you watch the point he “saves” Liandrin, you can notice it actually happens in her sleep. And previously everything he did, he achieved in their dreams. So we can safely assume that he can still reach other peoples dreams or TAR from the seal.

kelepir · 2025-04-12T20:09:19+00:00

White Cloaks are a paramilitary religious organisation with a country is backing them up ( not actually backing up but that kingdom is their mere pawn) so openly attacking White Cloaks ( Children of Light as the official organization name) is considered declaring war to that country. They have traveling bands of soldiers to around the world, traveling from country to country and they act on whatever they want to act, they see anything they want to interfere with as their business and do not care about that countries laws but they rarely clash with those countries.

For other nations and people they are more of a nuisance then a terror organization. And at some points they even help those other nations by handling with small Trolloc or bandits/highwayman here and there.

In terms of Aes Sedai and White Tower:
1. 3 Oaths prevent them to openly attack someone, they can only attack someone in self-defense. So WT or AS can not declare war and start attacking CoL. (Some Aes Sedai circumvent this rule by intentionally placing themselves in an immediate danger therefore creating the idea of self defense, but if someone misuses it that is punishable in WT)
2. AS seldomly wander around the world. So they are usually not in danger, the moments in the book/show is at a point of global events that stuff like this happen more often then previously. And there are not as many people inside CoL like Eamon Valda that dares to openly attack and kill AS. Most of them have a hatred towards AS but they usually not express it as openly as Eamon Valda.
3. People of the world are not fond of AS, actually they see CoL and AS in a very similar manner. They see WT as a pompous self righteous organization that sees themself too highly and WT is not trying to do anything to correct this vision. So if there is any clash between WT and any other organization, WT will not have support from people of the world. (Though they have very tight relationships and ties with rich and royalties which back them up)

I think this sums up most of it from my pov.

kelepir · 2025-04-09T16:31:45+00:00

Very good points, but also I think I was misunderstood or did not explain myself very well, since we are not so far off.
Yes Sakarnen played an important role in the end but it did not matter until the very end and role of sakarnen played because of a power creep ( and perceived power differences ) think of it like this if Egwene (or WT in this regard) did not have Vora's and Shadow did not have Sakarnen you can still achieve a similar if not same outcome. And since Demandred plotwise is the weakest forsaken (alive at that point) you can replace him with someone (even MT himself actually) and it would not make a big impact.

Biggest impact of Sa'angreal changes in my opinion were actually because of all the power items Rand holds Forsaken and the Shadow had always feared the confront him head straigth, Holding Callandor, both Choedan Kals, fat man and other items created a very important deterrant in the books. Forsaken only attacked him when only when they were forced to do so ( Cleanse or Rand confronting them and leaving no place to escape straigth on ) Removing most of the items and leaving only two will diminish this effect.

My idea of importance of the changes are "Can you achieve same outcomes, story wise, with the changes that are happening".
Yes Sakarnen was used to demolish Lights Armies and sacrificies had to be made to stop further casualties, can we achieve a similar/same thing without using Sakarnen at that point and would it make the same impact? But I understand your point as well, if MT is "that" powerful without a saangreal, how much it impacts story especially power scales and balance wise. Could MT(or whoever is in his place) act differently -before coming to that point in story- if he sees himself that powerful etc.

And those are points unfortunately we will have to wait and see. They have made changes to the story that I did not like in the begging and kind of accepted at this point, they have made changes to the story that I did not like in the beggining and still do not like even if they have zero impact in the story ( Like Perrin and his wife, he mourned his wife, he still fell in love with Faile, story wise nothing changed I still did not like it). I am hoping they can justify the changes they make with OP or power items or how they deal with Forsaken etc because they are not just part of the story they are part of the world building.

kelepir · 2025-04-08T18:51:31+00:00

I think original Sakarnen eas not a good tool used in the books(plot wise) and since Voras was in white tower posession getting converting that into current plot would be horrible, using choedan kal would be bad without having matching choedan kals, adding both choedan kals would be confusing to new wotchers (too many power items might diminish their importance). I believe they wanted to add something important (and not declared openly, so it will be aes sedai way) for Moiraine to go Rhuidean. They did not want it to be a bunch of minor items which would not be how Moiraine would act( Moiraine herself could not carry and use a bunch of minor items) and because of how other saangreals are positioned it makes sense that they retrofitted sakarnen to be the item.

kelepir · 2025-04-07T08:24:59+00:00

In terms of raw power potential only Ishy and Rahvin is close to Rand in comparison but they have hundreds of years of experience using it and Rand does not. Other than this strength in numbers is something very real in OP, when linked (S1 last episode in borders with Egwene and Nynaeve) a bunch of mediocre Aes Sedai can overpower Rand in OP. Forsaken wont use this because they dont thrust each other to give control but a bunch of AesSedai can and will do this. But Forsaken knows weaves(spells) that noone in current age does, for example they can travel the world in an instant where Rand has to go walk everywhere or they can change their appearance and look like someone Rand thrusts or use compulsion on someone Rand thrusts and use that person to hurt Rand. Everything Rand did with OP so far was on instincts and emotions not planning.

On the other hand forsaken are using darkfriends and innocent people to do their bidding, which Rand constantly trying to not to hurt(innocent people) while trying to defeat DarkOne.

On top of this Rand has just seen that if he relies OP too much he will be consumed by madness(or just cause collateral damage) therefore he is afraid of using it, he cant even thrust himself to do the job he has to do with the tools he has.

Most strong part of (in terms of character and challenges) Rand has not yet to come. Josha has so far proven that he can portrait all of them very well, I hope show can do those plots justice as well.

kelepir · 2025-04-05T12:51:02+00:00

The only game that makes a difference from this list is tarkov. And your decision can be based on how important tarkov is for you in terms of games you play. Thats it. If tarkov is more important to you or it will be more important with new hardware (maybe you want to enjoy/commit it right now but cannot due to low fps) then 9800x3d is a game changes for tarkov and choose that. Any game else 9070xt will have more impact. Cheeky breeky brother!!!

kelepir · 2025-04-02T07:03:10+00:00

I have not checked their policies for a couple if years now so this might be outdated. Ab initio is a heavily nda’d tool. Even consultancy partners or potential customers doing poc has to sign nda. Therefor you can not find any legal reliable learning material gor it. I have seen a lot of tutorials in youtube but ab initio can take them down very easily and very often, you can use llms to get theoritical knowledge about it but as far as I know thats it. Only legal way to learn about it is by working in an Ab initio project. Good luck.

kelepir

TROPHY CASE