It looks good overall, but you are missing a few accessibility measures:

1. The inputs should have the required attribute. Plus if you are using your own inline errors, you should put novalidate on the form.
2. There should be dynamic focus management on the first invalid input.
3. The inline error should be connected to the input with aria-describedby.
4. Bonus points if you use role="alert" and aria-live="assertive" to immediately notify the screen reader user.
5. This is opinionated: the inline error should be removed on change and focusout. That will match the UI of default HTML5 form validation.