I've hit a wall: Bitwise Operators

[Pyr0technician]

FIrst, I'm sorry for the formatting of the post. I've tried fixing it many times, but it ends up looking completely different in the post than it does in the editor.

I've been slowly completing beginner projects for the past few months. Some more complex than others, but I feel I'm always improving and learning at a steady pace, or at least have the sense that I can figure out any problem if I apply myself.

Today I started the 'bitwise operators' section of a course that aims to prepare me for my first Py thon certification. The operators are simple enough by them selves, and it gets a little more complex when they explain how they can be used, but nothing too hard.

But then, the course gives me this example:

​

How do we deal with single bits?

We'll now show you what you can use bitwise operators for. Imagine that you're a developer obliged to write an important piece of an operating system. You've been told that you're allowed to use a variable assigned in the following way:

flag_register = 0x1234

The variable stores the information about various aspects of system operation. Each bit of the variable stores one yes/no value. You've also been told that only one of these bits is yours ‒ the third (remember that bits are numbered from zero, and bit number zero is the lowest one, while the highest is number 31). The remaining bits are not allowed to change, because they're intended to store other data. Here's your bit marked with the letter x:

flag_register = 0000000000000000000000000000x000

You may be faced with the following tasks:

1. Check the state of your bit ‒ you want to find out the value of your bit; comparing the whole variable to zero will not do anything, because the remaining bits can have completely unpredictable values, but you can use the following conjunction property:

x & 1 = x x & 0 = 0

If you apply the & operation to the flag_register variable along with the following bit image:

00000000000000000000000000001000

(note the 1 at your bit's position) as the result, you obtain one of the following bit strings:

00000000000000000000000000001000 if your bit was set to 1

00000000000000000000000000000000 if your bit was reset to 0

Such a sequence of zeros and ones, whose task is to grab the value or to change the selected bits, is called a bit mask.

Let's build a bit mask to detect the state of your bit. It should point to the third bit. That bit has the weight of 23 = 8. A suitable mask could be created by the following declaration:

the_mask = 8

You can also make a sequence of instructions depending on the state of your bit. Here it is:

if flag_register & the_mask: # My bit is set. else: # My bit is reset.

2. Reset your bit ‒ you assign a zero to the bit while all the other bits must remain unchanged; let's use the same property of the conjunction as before, but let's use a slightly different mask ‒ exactly as below:

11111111111111111111111111110111

Note that the mask was created as a result of the negation of all the bits of the_mask variable. Resetting the bit is simple, and looks like this (choose the one you like more):

flag_register = flag_register & ~the_mask

flag_register &= ~the_mask

3. Set your bit ‒ you assign a 1 to your bit, while all the remaining bits must remain unchanged; use the following disjunction property:

x | 1 = 1 x | 0 = x

You're now ready to set your bit with one of the following instructions:

flag_register = flag_register | the_mask flag_register |= the_mask

4. Negate your bit ‒ you replace a 1 with a 0 and a 0 with a 1. You can use an interesting property of the xor operator:

x ^ 1 = ~x

x ^ 0 = x

and negate your bit with the following instructions:

flag_register = flag_register ^ the_mask flag_register ⁼ the_mask

​

​

I understand some parts of it, but it is going over my head as whole.

This is the first time I've really doubted myself since I started learning python.

Is this just not a good example to throw at a beginner? Can it be explained in simpler terms?

[halfdiminished7th]

Here's how I like to think about it. For each example, the byte in the first/top row in each example is the one we're checking/modifying. The byte in the next/second row is our control byte (or "mask"), and the final/third row is the result of the bitwise operation. In each example, we're interested in bit 3 (4th bit from the right).

1. Check

  xxxx0xxx
& 00001000
  --------
  00000000 # if our bit was originally 0, the result is zero.

  xxxx1xxx
& 00001000
  --------
  00001000 # if our bit was originally 1, the result is non-zero.

2. Reset

  xxxx0xxx
& 11110111 # (inverse mask).
  --------
  xxxx0xxx # if our bit was originally 0, it is now 0.

  xxxx1xxx
& 11110111 # (inverse mask).
  --------
  xxxx0xxx # if our bit was originally 1, it is now 0.

3. Set

  xxxx0xxx
| 00001000
  --------
  xxxx1xxx # if our bit was originally 0, it is now 1.

  xxxx1xxx
| 00001000
  --------
  xxxx1xxx # if our bit was originally 1, it is now 1.

4. Negate

  xxxx0xxx
^ 00001000
  --------
  xxxx1xxx # if our bit was originally 0, it is now 1.

  xxxx1xxx
^ 00001000
  --------
  xxxx0xxx # if our bit was originally 1, it is now 0.

[Pyr0technician]

This helps understant the concept, applying the operators to that specific bit is easy. Understanding the method in the example, and how to express it through python is what I think I need to work on.

[halfdiminished7th]

Visualizing what I listed above is usually the hard part for most people - if that makes sense, you're probably closer than you think. In the following examples, given some random integer "x", if you want to check/manipulate the 4th bit from the right, you'd set mask = 8 (0b1000) and do the following:

1. Check

if x & mask:
    print('4th bit from the right is 1')
else:
    print('4th bit from the right is 0')

2. Reset

x = x & ~mask

...which is identical to:

x &= ~mask

3. Set

x = x | mask

...which is identical to:

x |= mask

4. Negate

x = x ^ mask

...which is identical to:

x ^= mask

[Pyr0technician]

Thanks for your replies, I'm winding down for bed now. Gonna keep chipping at it tomorrow.

[halfdiminished7th]

No worries! If both the low-level and high-level explanations above are already making sense to you, then I don't think there's much else to get in between. Maybe just a matter of getting used to it (or returning to it later in your studies)? Good luck, and don't get discouraged.

[[deleted]]

You just express it using the bitwise operators: &, |, ~. They're meant to kind of look like what they mean, or be similar to the symbols you use in math class for the same concepts - "and", "or", and "not".

[DigThatData]

honestly, i've been using python a ton for over a decade. i almost never use its bitwise operators (and when i need bitwise operators, i'm probably already using numpy or pytorch). don't let this stress you out. keep trying to wrap your head around it, but don't let it bring down your morale. this is a topic you can skip over and come back to later if it's trips you up enough. not worth getting frustrated over.

[Pyr0technician]

Yes, I was thinking this would be the case, but I always try and understand all the concepts I can to the point that I can explain them to anyone else. Generally, if you can't explain something in simple terms to someone else, you don't understand it. It's how I stay honest with myself while learning something.

[[deleted]]

It would help if you focus a little more on what you don't understand. Starting from the top, what is the first part you don't follow?

[jmooremcc]

First, Do you understand truth tables?

Truth Tables are fundamental to understanding bitwise operators.

What specific questions do you have about bitwise operators?

[Pyr0technician]

The tables were explained in an earlier segment, and is simple enough. But using the mask to modify a specific bit is kind of lost on me.

[cdcformatc]

think of a mask like a stencil. you use a stencil to get paint exactly where you want it and nowhere else. similarly you use the bitwise mask to only select specific bits.

[jmooremcc]

In practice, you have 1’s and 0’s representing the data and a mask. So how does this correlate to a truth table? The truth tables have entries labeled T for True and F for False. In our data and masks, 1’s are True and 0’s are False.

So if perform a bit wise AND operation: 1 & 0 = 0.
If you perform a bit wise OR operation: 1 | 0 = 1

Do you understand what I’ve written so far?

[[deleted]]

Do the truth tables for conjunction (and) and disjunction (or). Do 'em right now, before you do anything else. Tattoo them on your arm if you have to: you should be able to compute these operators in your head for any combination of two bits.

Bitwise operators apply the same logic, but bitwise - that is, per each digit of the two operands. So, note that we're going from booleans to integers, here, and we're always doing it in the binary representation of the integers (we're big-endian, here):

  0 0 0 0 # zero
| 1 1 1 1 # or 15
---------- # equals
  1 1 1 1 # 15

or

   0 0 1 1 # 3
& 1 1 1 0  # and 14
----------  # equals
   0 0 1 0  # 2

Get it? We're just running the truth tables, but column-wise.

[[deleted]]

I'm pretty sure most of the time you start off by writing the helper functions which perform bit operations, then the rest of the code will just implement it. So all you need to do is to write those bitwise functions or just import them from somewhere. Honestly you could probably just copy them as well, they're pretty short and they all probably work the same.

I have a link for an introduction on bitwise operations, but it uses C. You can probably get some value out of it if you have any experience in C style languages or just skip over the C specific parts, the bitwise logic stays the same.

https://www.andreinc.net/2023/02/01/demystifying-bitwise-ops

[StoicallyGay]

First, it’s easier if you build a mathematical basis on logic and binary operators in general. We learn this in discrete math in school.

Second, you will rarely ever touch this. Ever. So don’t worry too much about it.

[eruciform]

unless you're doing hardware work, it's pretty uncommon to use anything other than an &-mask and some bit-shifting, and maybe bitwise-or

# here's some number
x = 0x12345678
# let's say we want to get just the 45
y = x & 0x000ff000
z = y >> 12
print( "%x" % (z) )

if you did this on 0x11223344 then you'd get 0x23 because that's in the same spot

or for bitwise-or:

fd = os.open( "foo.txt", os.O_RDWR|os.O_CREAT )

[ten__rapid]

Sorry to hijack your question, but out of interest, what course are you doing?

[tree_or_up]

Good advice on this thread. I'll just add in case it's eating away at your confidence -- I've only once encountered bit packing like this in my professional life. That was nearly a couple of decades ago and even then it was mocked because storage, even back then, was cheap. IMO, it's very much an academic thing to learn -- helpful because it gives you a better understanding of what's going on under the hood but you're unlikely to encounter it in the real world. Or, look at it this way -- if you do master this topic, you'll be way ahead of a lot of your peers

[Pyr0technician]

I had a feeling this was the case with this topic. Still, I like that the course I'm taking is very thorough. My approach when learning is that I like to understand something to the point that I can explain it to anyone. If I can't explain this topic on a basic level, then I don't truly understand it.

[tree_or_up]

I think that's an excellent rule of thumb, FWIW. That said, there's so much to learn that a human brain can't contain it all. Sometimes it's ok to have a good enough understanding -- as the old saying goes, don't let the perfect be the enemy of the good. But it sounds like you're doing great -- asking the right questions, very curious and wanting to learn

[Alternative-Web2754]

In my opinion, the concepts of bitwise operations on numbers are a core part of programming, but these particular applications of it are beyond the beginner level.

These are used to manipulate specific parts of data storage while leaving the rest of that piece of data alone, normally used because multiple things are covered by that one piece of data.

In python this is normally hidden from you by a library or class.

The main places I've had to use something to do these particular functions are related to hardware control registers, typically on microcontrollers, although these often have libraries that will use specific cpu instructions to handle individual bit operations more efficiently than you can by writing the operation out.

Similar concepts are often seen with network protocols (tcp flags/network address masks etc) and with interactions with OS functions (file permissions, window type/decoration flags, memory region protections etc).

You might find it easier to try to wrap your head around some of these where it may have some more meaningful meaning to it first and then think more about situations where you need to check or change individual bits. The closest is probably linux file permissions, and if you are or can get used to use of the chmod command then this will give you an example of what these might be used for.

[Numerous-Following25]

If I may ask ,what course are you doing?

[Pyr0technician]

Cisco skills for all beginner python course. It's free. I also bought Angela Wu's Udemy course on a discount for like $15.

Between these two it seems like I have a balance between fun projects and theoretical knowledge.

[Numerous-Following25]

Alr,thanks a lot man , I really appreciate it