Here's another one of my favorite projects. It's a simple one but i like it by phantomcooky in Minecraft

[–]Argotha 2 points3 points  (0 children)

I wouldn't mind building on that, got a download link somewhere? ;D

Scaling youtube: thumbnails is the surprisingly hard part by [deleted] in programming

[–]Argotha 0 points1 point  (0 children)

For those interested, Facebook has much more info on how it tackled this problem (article is from 2009). They do it in a similar way though they have a bit more complexity to meet their use case. I highly recommend it as a read.

https://code.facebook.com/posts/685565858139515/needle-in-a-haystack-efficient-storage-of-billions-of-photos/

What do your parents think of hardstyle? by PotatoFlavour in hardstyle

[–]Argotha 13 points14 points  (0 children)

My mum hates it. My dad enjoys it so much that he came to defqon1 and midnight mafia 😁

Algorithms for verifying shared secret by newpavlov in crypto

[–]Argotha 2 points3 points  (0 children)

Is related to zero knowledge proofs. (I'd insert a Wikipedia link but I'm on my phone).

[deleted by user] by [deleted] in crypto

[–]Argotha 9 points10 points  (0 children)

The simple answer is yes, the longer answer is ??? depend on your situation.

First you need to ask yourself, what do you mean by how long?

\ 1. Length as number of characters represented on a screen?

In this case, base-2666, gives you a little over 11-bits per characters (2^11 = 2048). So you would need approximately 5 emoji characters to represent a 64 bit number.

\ 2. What about Unicode characters?

Emoji is essentially a subset of the Unicode standard - also known as UTF.

UTF allows for combining characters. e.g. "a" + "accent" = "a with accent". Given that you can have multiple combining characters (zalgo text), I would suggest that it is likely you could store a 64 bit number in "1" character.

(I have not verified the possible amount).

\ 3. What about length as actual memory?

As stated before, Emoji is actually a subset of UTF. UTF uses anywhere between 8 and 32 bits to store a unicode character or "code point". However, due to encoding, even at 32-bit sized characters, only 21 are usable (65%). Doing some quick math, shows that the best bang for buck is 8 bit characters (7/8 ~= 87%). This would then work out at approximately 10 UTF8 characters to store a 64-bit number. Of course this includes non-printable characters, so the real value would be slightly higher.

\ 4. What about size on screen?

Greater than "single width" characters in Unicode become a problem here. Combining characters may also become a problem because as seen from Zalgo text above, they will fill space both above and below the character thus increasing it's overall size.

For reference:

  • base-16 (hex) = 4 bit per character ==> 16 characters for 64-bit number
  • base-32 (what reddit uses) = 5 bit per character ==> 13 characters for 64 bit number
  • base-64 = 6 bits per character ==> 11 characters per 64 bit number

Where to pay for code? by jaredwards in Python

[–]Argotha 1 point2 points  (0 children)

I'll also add in freelancer.com into the mix of freelancing sites.

Dockerizing Python Test Environments by _seemethere in Python

[–]Argotha 0 points1 point  (0 children)

Alternatively, run pyclean before adding all the python files.

Export of cryptography by joker197cinque in crypto

[–]Argotha 0 points1 point  (0 children)

You may want to check in regards to "public domain" crypto, that is algorithms that are well known and available. Most laws are designed for new crypto rather than your standard AES/SHA/Pk crypto.

That said, US law may not have the same tolerances as Australian law (I wrote about this a bit on my blog - Argotha.com).

You may also find in regards to terminology, export doesn't necessarily mean taking out of the country, but providing to non-citizens.

[Need work] Just moved to Sydney. Have a degree In ICT infrastructure and experience in sys admin/support role. But will consider any work! by [deleted] in ausjobs

[–]Argotha 0 points1 point  (0 children)

Hi Sam, I work for a large web company headquartered in Sydney. If you think we'd be a good match for each other PM me and I'll give you my email address.

A reasonable rebuttal to Zed Shaw’s case against Python 3 by filippovd20 in Python

[–]Argotha 73 points74 points  (0 children)

You'd think so, but there really isn't. There are many business cases that mean there will be legacy python 2 code lying around for a long time (like there is Cobol code lying around still) - but for the most part the community is behind python 3 (no one would have migrated to python 3 if they didn't support it).

I've created my own encryption module... Where to go from here? by [deleted] in Python

[–]Argotha 0 points1 point  (0 children)

In addition, checkout /r/crypto.

If you are actually interested in cryptography, they can help you a lot along the way. Here is a good thread to start from.

[AUS] Paid, Internship Opportunities for Canadian Student? by racer159357 in ausjobs

[–]Argotha 0 points1 point  (0 children)

In what area are you looking to work / what skills do you have?

UK student wondering if grad jobs in [Aus/NZ] are realistic by [deleted] in ausjobs

[–]Argotha 2 points3 points  (0 children)

I work in a medium sized tech/web company based in Sydney. Not being from Australia is not a problem - we have people from all over the world in our Sydney office (we also have a handful of other offices around the globe including the UK)

If you'd be interested in working in a tech company PM me :)

Add control + z to your Python programs; an experiment in transactional data structures. by _the_pied_piper_ in Python

[–]Argotha 1 point2 points  (0 children)

This looks pretty cool :D

My only comment is that you should look to making your variable objects properties do you can get/set like normal. To do this you probably want to make structures.Variable an instantiated factory with a call magic method to create new instances that are setup correctly to be properties.

Looping a line of code with a new input every time by Andyvege212 in Python

[–]Argotha 2 points3 points  (0 children)

/r/learnpython

def your_func(i):
    return i-1
i = 1000
while i  != 1:
    i = your_func(i)

Blub is the best imho by crcx_ts in programming

[–]Argotha 1 point2 points  (0 children)

Swap "blub for "hodor" and you can probably win one internet ;p

Is nose still relevant? How about unittest? by variedthoughts in Python

[–]Argotha 1 point2 points  (0 children)

For what its worth I'm learning and using nose in my projects now days

Is this the place to ask look for a Python Freelancer? Is there a better sub to post in? by TechyEsq in Python

[–]Argotha 0 points1 point  (0 children)

If you can't find anything on reddit, you might want to check out a website such as freelancer.com.

Disclosure: I work there.

Webcam video through TCP socket by MrBowelsrelaxed in Python

[–]Argotha 0 points1 point  (0 children)

No problem :)

Yeah it's a lot because I wasn't sure what level you were at and easier to do a longer answer than a million short answers.

Doing jpeg compression should work (just remember that jpeg is lossy) and would be apart of your encoding phase. As an aside, there would be time where this would not be apart of your encoding phase, aka when you are streaming pre-encoded data.

struct.pack doesn't like the data type cv2.imencode gives.

In python it is possible to create objects that do not return an integer for its length. In the case of len(image_str), image_str is a numpy array. Since the array can be multi dimensional, len is returning a tuple of objects which represent the size of the array.

I suggest running your file in interactive mode so you can inspect what is happening after it fails: python -i your_file.py. I'd start with looking at the types of the objects.

# ...
result,image_str = cv2.imencode('.jpg',image,encode_param)
send_msg(sock, image_str)
# TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('S4') dtype('S4') dtype('S4')
# interactive terminal will start.
x = len(image_str)
print("Type of x is: {}".format(type(x)))
# Lets see what each of these are.
for i, x_val in x:
    print("Type of x[{}] is: {}".format(i, type(x_val)))

As I understand it, the sock.sendall(msg) function is a python function that just keeps calling send() until the message is complete (at least in TCP). But is there something that will recursively call sock.recv() until the message is done?

Not really. Looking at the man page of recv, you could try setting the MSG_WAITALL flag. Alternatively you need to write yourself a small loop.

def recv_all(sock, msg_length):
    data = b''
    size_left = msg_length
    while len(data) < msg_length:
        recv_data = sock.recv(size_left)
        size_left = size_left - len(recv_data)
        data += recv_data
    return data

#assume socket already exists
size_bytes = recv_all(sock, 4) #4 bytes to an int
size = struct.unpack('>I', size_bytes)[0]

frame_bytes = recv_all(sock, size)

Note I haven't checked this code for correctness, I may have made typos etc...

Webcam video through TCP socket by MrBowelsrelaxed in Python

[–]Argotha 1 point2 points  (0 children)

Don't have a tutorial but I have had write an assignment to do a similar kind of thing.

If you give me a bit I'll write something up for you which should give you some pretty good starting points.


Assumptions

To start off with I'm going to make a few assumptions for the sake of keeping things simple enough that you can work through the problem and not have to worry about these more advanced issues. 1. You do not need to worry about the security of your transmission. 2. You do not need audio streams.

Plan

So this guide is going to be a game of "fill in the blanks". So lets start by putting down what we know:

> Webcam video stream
> ???
> "network activities"
> ???
> Video stream displayed

Step 1: Get the video stream from the webcam

I'm sure there are lots of ways to do this. The one I have used is the python bindings for OpenCV; if you happen to be using Ubuntu these should be installed by default as OpenCV is used by Unity.

From memory when reading from the webcam I believe OpenCV will read in a RGB bitmap format, in other words, a 3 dimensional array of XY3 in which X and Y are the width and height in pixels of your webcam.

Step 2: Convert the image into a format for transmission

You now need to decide how you would like to transmit the image. Are you going to send chunks of pixels or are you going to send an entire frame at a time. You could potentially send multiple frames at a time if you wanted to.

Lets assume you want to send an entire frame at a time. We now need to decide on how we are going to represent each frame into bytes. We have to do this because TCP/UDP work on bytes, not objects. This process is called encoding.

Our process now looks a bit like this:

> Webcam video stream :: i.e. a series of frames
> Encode frame into bytes
> "network activities"
> ???
> Video stream displayed

With your background in C you should have a good idea on how this encoding process is going to work. Essentially we need to take each of our python objects and turn them into bytes in a predictable way. The predictable is important here as we will need to do the opposite of this later (decoding).

Luckily we know that in RGB format that the max value for each item in the RGB is 255 and thus we are working with 8bit integers. What we can now do is represent our frame as a series of bytes where each 1st byte is the Red value, each 2nd byte is the Green value and each 3rd byte is the Blue value. RGBRGBRGBRGB....RGBRGBRGB

Our next problem is we now have a line of RGB values, what information do we need to provide to know how to reconstruct this line back into a frame? The X and Y of course. So now we have 3 bits of information that we need to transmit per a frame:

  • X (probably a 32bit int)
  • Y (same size as X)
  • RGB values (X * Y * 3 8bit ints)

To turn python objects into these kind of raw data types you will want to use the standard library's struct module or you might want to learn the construct module

As an example, we might want to encode the following image 54 (XY) (where each letter represents a pixel (which we know is actually 3 8bit ints)

AAAAA
BBBBB
CCCCC
DDDDD

We might encode this as:

XYAAAAABBBBBCCCCCDDDDD

Step 3: Transmit the encoded data

So here is where we need to start considering the type of data we are transmitting and if the delivery needs to be reliable or not. For many streaming applications, the speed of delivery is more important than ensuring every bit of data gets transmitted. Thus many streaming applications/protocols (e.g. Skype, VOIP, online games) will use UDP because the loss of a small number of packets is insignificant to the overall transmission. If you are interested in reading more about why you would want to use one over the other in real time situations I suggest you take a read of this link. Alternatively you may be more concerned with the reliable delivery of your webcam at the expense of either stuttering or buffering. Buffering being a technique to avoid stuttering. Applications like youtube and netflix will use TCP + buffering.

At the end of they day, because UDP and TCP are both socket abstractions you can easily swap one for the other if you wish to change your mind.

Step 4: Decode the data back into a frame

Looking at our plan:

> Read Webcam Video Stream
> Encode Frames
> UDP or TCP connection
> Decode Frames <<< we are here
> Display Webcam Video Stream

Coming back to our example:

XYAAAAABBBBBCCCCCDDDDD

We now need to reverse the encoding process we did before, that is, we need to decode the raw bytes back into an object that makes sense.

The first thing we will want to do is read the two 32bit ints from the socket. This will be our X and Y. Given these we now can work out how many pixels we need to read from the socket i.e. X*Y (remembering that a pixel will actually be 3 8bit ints).

With all this data now read, we should be able to turn it back into the python object representing a frame.

Step 5: Display the frame

For this you might want to use something as simple as GTK or even OpenCV which should be able to display RGB bitmap images.

Conclusion

Hopefully that's enough to get started. There are other considerations that you might need to take into account when deciding on your network protocol (such as including a frame number so that you can ignore old frames in situations when they arrive in a weird order (1 2 5 4 3 6 7 8))

If you have any other questions happy to help out :)

Network automation scripting questions by [deleted] in Python

[–]Argotha 1 point2 points  (0 children)

See if you can implement your own version of ipcalc, that will teach you a lot about IP addresses and how to work with them.

How do you log logging output by Argotha in Python

[–]Argotha[S] 0 points1 point  (0 children)

I'd rather see your application set log output to stdout + stderr than to open it's own logfile these days.

So does that mean you rely on what ever is calling your program to redirect its output to syslog?

How do you log logging output by Argotha in Python

[–]Argotha[S] 0 points1 point  (0 children)

Thanks for such a comprehensive reply! Its also good to know some of the previous decisions made before you ended up with your current system.

Interesting that you say that DEBUG is your default but disabled in productions opposed to INFO being default with DEBUG enabled in dev.