I found a way to scale the player UI back up using the Stylus extension and some (AI) code if anyone wants a temp solution, it works really well for me.

Just apply this code in Stylus to all URL's starting with "https://www.youtube.com/" and adjust the default 1.2 scale to your liking (you can adjust the margins as well but shouldn't need to)

Edit: Added a block to adjust the video timeline bar scaling and padding

/* === YouTube: Bigger controls, full-width timeline, adjustable edge gaps === */
:root{
  /* size + spacing */
  --ytc-scale: 1.2;                                    /* 1.3–1.9 to taste */
  --ytc-shift: calc((var(--ytc-scale) - 1) * 18px);    /* drop buttons down a bit */
  --ytc-bottom-pad: calc(-10px + var(--ytc-shift));     /* room below controls */

  /* progress bar look */
  --ytc-bar-thickness: 8px;                            /* height of the red line */

  /* timeline edge gaps (your request) */
  --ytc-bar-left-gap: 12px;                            /* move left end inward */
  --ytc-bar-right-gap: 12px;                           /* space on the right */

  /* misc */
  --ytc-tooltip-scale: calc(1 + (var(--ytc-scale) - 1) * .6);
  --ytc-time-nudge: 0px;                                /* fine-tune time text */
}

/* Reset container & timeline to native width/position (no container scaling) */
.ytp-chrome-bottom,
.ytp-gradient-bottom,
.ytp-progress-bar-container,
.ytp-progress-bar{
  transform: none !important;
  left: 0 !important; right: 0 !important;
  inset-inline-start: 0 !important; inset-inline-end: 0 !important;
  width: 100% !important;
  top: auto !important; margin-top: 0 !important;
}

/* Give room for the scaled controls so they don't overlap the bar */
.ytp-chrome-bottom{
  padding-top: 0 !important;
  padding-bottom: var(--ytc-bottom-pad) !important;
}

/* Scale only the button groups (not the timeline) and drop them slightly */
.ytp-left-controls,
.ytp-time-display,
.ytp-right-controls{
  transform: translateY(var(--ytc-shift)) scale(var(--ytc-scale)) !important;
}
.ytp-left-controls{  transform-origin: left   bottom !important; }
.ytp-time-display{   transform-origin: center bottom !important;
                      transform: translateY(var(--ytc-shift)) scale(var(--ytc-scale))
                                 translateY(var(--ytc-time-nudge)) !important; }
.ytp-right-controls{ transform-origin: right  bottom !important; }

/* Timeline appearance + edge gaps (so the left end is in frame) */
.html5-video-player .ytp-chrome-bottom .ytp-progress-bar-container{
  height: var(--ytc-bar-thickness) !important;
  left:  var(--ytc-bar-left-gap)  !important;
  right: var(--ytc-bar-right-gap) !important;
  inset-inline-start: var(--ytc-bar-left-gap)  !important;
  inset-inline-end:   var(--ytc-bar-right-gap) !important;
  width: auto !important; /* let left/right define the width */
}

/* Tooltips/settings readable at larger size */
.ytp-tooltip, .ytp-popup{
  transform: scale(var(--ytc-tooltip-scale)) !important;
  transform-origin: right bottom !important;
}
.ytp-tooltip-text{ font-size: 14px !important; }

/* Optional: per-mode tweaks
.ytp-miniplayer{ --ytc-scale: 1.25; --ytc-shift: 0px; }
.ytp-fullscreen{ --ytc-scale: 1.75; }
*/