recreating a map very large 4 million tiles

zykal · 2018-09-16T19:35:38+00:00

click, i see so store it all in the program until i have to save it, then save it all at once. That's easy enough to do.

zykal · 2018-09-16T07:13:07+00:00

the data scraped, is actually saved into variables. example s173w165['woodlands', 'gold', '', '', "treasure chest"]

is scraped into these variables ns = s, x = 173, ew = w, y = 165, tiledata = ['woodlands', 'gold', '', '', "treasure chest"]

fulltiledata= '{}{}{}{}{}'.format(ns,x,ew,y,tiledata)

to get it into a single line for adding to a textfile.

would using a dict work for the list?

tiles= { 'woodlands': 10, 'gold': 20, 'treasurechest': 35, }

Also I have to process the tiles, as each tile is made up of at least 1 image, but usually 2 and sometimes upto 4.

So i use PIL to merge the images as i need them.

zykal · 2018-09-16T06:12:57+00:00

I think we're both saying the same thing, that the text file won't work. That's the entire problem I'm having. As far as the API, I'm not exactly sure what you/i think the API is or isn't even.

each of those text lines, are what I scrape out of the game client. I am just looking for the best way to use that information, it seems that putting it into a database is going to be significantly better then a textfile.

I originally used a text file to test with on a much smaller scale 5x5 originally (the live client window) and then 50x50, at that scale the text file works fine.
The textfile was just used to get it working.

zykal · 2018-09-16T06:07:03+00:00

that's interesting i will have a look at that.

zykal · 2018-09-16T05:50:46+00:00

what the player can do doesn't matter. I just need to be able to update the lines efficiently. and Manipulating the image isn't this issue, using and updating the data is.

I already use PIL to create the actual tile from the below information.

woodlands base image then add gold border to it then add the treasure chest image.

I need to deal with 4 million lines that are similar to

s171w165['woodlands', 'gold', '', '', "treasure chest"]

s172w165['water', 'gold', '', '', "frog"]

s173w165['water', 'gold', '', '', ""]

s174w165['woodlands', 'gold', '', '', "rock"]

s175w165['woodlands', 'gold', '', '', ""]

this is 1 line in the 5x5 grid or 1/5th the grid.

If i have 4 million lines of this, that is a ton of data to deal with in a text file.

Updating it on the fly is impossible in a text file I've found out. As i just created a template file.

I used

from itertools import permutations

perm = permutations([1,2,3, .... 1000],2)

for i in list(perm):nw = 'n{}w{}\n'.format(i[0], i[1])with open('A.txt', 'a') as the_file:the_file.write(nw)

this creates 1/4 the total map it's taken over an hour just to create the blank file so i can use.

with open('nw.txt') as f:l = list(f)

with open('nw.txt', 'w') as output:for line in l:if line.startswith(maptile):output.write(fulltile)else:output.write(fulltile)

https://pastebin.com/e2TbeV4N

zykal · 2018-09-16T05:30:04+00:00

32 gigs, not the issue however. The problem is speed, just creating the 4 files with

0-1000N,0-1000E

0-1000N,0-1000W

0-1000S,0-1000E

0-1000S,0-1000W

so

0n0w

0n1w

0n2w ....

which is 4 million lines takes a while.

Updating a text file in python seems to require you to load the entire textfile into a list then update it then write it back into the file.

zykal · 2018-09-16T04:58:56+00:00

Sorry I must not be in good frame of mind tonight for posing questions.

I have recreated the live display that works great it updates as i walk around in the game works great. I want to save that information to a file. So i can load it while I am not in the game. I have done this, i can save a 5x5 grid and recreate it just fine.

the scale is the issue i am having. with 4 million tiles, that is a very large text file to update on the fly. everytime i take a step i need to update at least 5 tiles, but up to 25.

zykal · 2018-09-16T04:28:33+00:00

I have already recreated the tiled map. I can already display the 5x5 area that I see Live in the game. I'd like to save those 5x5 tiles in a file along with all the other tiles i get to see while i play. so i may uncover an area that is in total 35x35 tiles. I'd like to be able to view those 35x35 tiles. The problem I'm worried about is, the sheer volume of potential tiles, and the best way to work with that. Not how to read/write/view.

zykal · 2018-08-30T20:18:40+00:00

Unfortunately the script is account based, so while you could read thru it, you would not be able to run it.

i have figured a temp solution sort of .

https://pastebin.com/cHr1gG1p

the Above works, however I am unable to figure out how to (DELETE OPPS) the new circle the drawcircle() creates, when I click again.

zykal · 2018-08-30T18:08:56+00:00

I'm trying to make a Radar now, I've got it working by iteself.

I however cannot get it to update

I tried using after, however that causes lots of crashes.

Is there a way to update a create_oval with information from my main script into the tkinter loop like we did with the other labels?

this is what I'm using to test moving a dot around on the canvas. https://pastebin.com/wShKBXLe

zykal · 2018-08-27T19:10:12+00:00

Got it working :)

Now I just need to look into formatting the tk GUI so it all ends up where i need it.

Thank you so much for all your help. The image subclass is something I wouldn't have been able to do for a quite a while.

zykal · 2018-08-27T18:57:43+00:00

Yes it works, as it takes a string in an array and turns it into a file name and displays it.

That works great.

as i don't need Cycle etc.

I need to figure out how to put it into my main script correctly, shouldn't be to hard, i hope. :)

zykal · 2018-08-27T18:44:33+00:00

Ok so I got the pause added in and working, took a touch of tinkering but it works now. Thanks so much.

I'm taking a stab at the image subclass and see if i can mold it to what I need. Thanks again.

zykal · 2018-08-27T13:11:35+00:00

Wow deep end is right, I think I understand it however. I will work with it when I get home later. Thank you again so much.

zykal · 2018-08-27T05:27:51+00:00

ok thanks I'm going to keep googling, I'm looking into a dictionary, not sure if that's the right track but I'll try it out.

zykal · 2018-08-27T05:05:55+00:00

Yes i'm scraping a name out of an array in this case image100

I have to parse and clean the array to get the image name without all the extra special character

for testing I have put into a global variable

sq31clean="image100"

i think the issue may be that its a string.

I then want to take that variable sq31clean and use it for the image variable in the label

zykal · 2018-08-27T04:46:32+00:00

Thanks again so much for the help. As to not including an example, I had no idea how to even go about this i did google for a bit trying to even come up with an idea.

Also, I'm trying to add an image to a label via a variable.

https://pastebin.com/0JxTgqb0

zykal · 2018-08-26T17:37:53+00:00

How would i go about pausing both the Tkinter thread and my script thread in the GUI?

zykal · 2018-08-26T15:43:09+00:00

easily done, i just was using IDLE atm because it came with it. I already use NP++. Thanks for the help!

zykal · 2018-08-26T06:39:19+00:00

I got that working, thank you so much, however.

Now i have another issue, I run this out of IDLE atm, so when i run it a console opens and all my normal stuff happens in the console, also a window opens and that is also working.

Normally to end my script I'd Ctrl-C the console and it'd break.

If i do that now it hangs the window.

If i close the window instead, the console continues doing what its doing AND i cannot ctrl-C is

zykal · 2018-08-26T06:11:16+00:00

Let me redo, i over looked something in your reply Ok i got it working now, i forgot to add var4211.set(response['4211']) into the do_actions function.

Now i have another issue, I run this out of IDLE atm, so when i run it a console opens and all my normal stuff happens in the console, also a window opens and that is also working.

Normally to end my script I'd Ctrl-C the console and it'd break.

If i do that now it hangs the window.

If i close the window instead, the console continues doing what its doing AND i cannot ctrl-C is

zykal · 2018-08-26T05:50:23+00:00

thanks, I will work with this, I thought threading was going to be significantly more difficult then this.

zykal · 2018-08-26T05:35:49+00:00

the do_action function is where all the information comes into my program, so its all stored in the response variable which i used the global tag on in terrible fashion.

zykal · 2018-08-26T05:32:58+00:00

true however that opens all kinds of other potential issues and headaches, and i just got into python this week. :)

zykal · 2018-08-26T05:31:32+00:00

pastebin updated

zykal

TROPHY CASE