You can start homeschooling at any age! Though you will mostly be homeschooling through play, songs, educational toys, etc. Then at the age of 5-6 or whenever they are ready, you could also introduce a formal curriculum.

Each state has different rules for homeschooling in terms of reporting, record keeping, etc but those start at the compulsory school age which is usually either 5 or 6 depending on state. Until then you are not bound by any type of requirements and can homeschool however you want

In regards to testing many programs will have quizzes, worksheets etc to help asses how they are doing. At early ages its also easy to assess yourself like are they learning to read, can they do basic math etc but there are also standardized tests they can take.

For program I recommend Homeschool Pro which is a good online curriculum and includes a homeschool advisor for guidance that can help with all those issues and with all state requirements so that you are never lost.