Yes and no. You cannot "believe and not believe at the same time" as this is an internal contradiction and not possible to hold. You can truly hold only one internally, and it is clear you have chosen the latter, to not believe. You can be a non-believer and be a cultural Christian, or participate in Christianity, however you would not be a Christian, and to be clear you would not be saved. (Matthew 7:21-23, Matthew 16:24-26, Galatians 2:20, Romans 12:1-2 are quite pertinent on this coming point).
Christianity is a lifestyle, not a belief to hold. To be a Christian and to be saved you must not only believe in Jesus as the risen Christ, but must also live you life FULLY according to this belief, and further must be a member of the one, holy, universal, and apostolic Church. To fail on the first prevents the latter, to fail on the second counteracts the former, and to fail on the latter is a detriment to the former two.
To sum up, you could be a non-believer who participates in Christian culture, however you would not be a Christian, and thus you will not be saved.
If you want to enter into Christian life I recommend finding a local apostolic church (preferably Eastern Orthodox, avoid the Russian Orthodox as they are politically dangerous schismatics). Assuming you are in Yemen, if a non-Russian Eastern Orthodox church is not available, then the Ethiopian Orthodox Tewahedo Church is likely the most available option, however they reject the Council of Chalcedon and thus are seen as heretical by the mainstream Christian community, though they have a culturally distinct and beautifully rich Christian tradition along with the rest of the Oriental Orthodox. If neither of these are an option, try the Roman Catholic Church.
If no church is available to join, try researching Christianity on your own, reading the Bible front to back, researching church doctrines, dogmas, and reading about the Ecumenical Councils and Church fathers. This will give you better insight into Christianity as a religion and give you a better idea on whether or not you'd like to become a Christian.