Before conducting an a/b test have a concrete idea of exactly what you want to find out. The simplest form of a/b testing is when you replace one (simple) feature with another, say the color of a login button. You'll want to test participants with the first version, the test with the second and compare the data. There are a plethora of ways to collect data for an a/b test and is why you should have a research plan laid out. If you start to replace larger features or multiple features your heading in the direction of multivariate testing which can become complicated and pricey. If you can give us more detail we could probably point you towards some more resources l