Gammerz you are correct but there’s a piece of the puzzle missing - the universe was still really big even back when the first stars formed and stars formed in it at random places in random order. So there are certainly some early stars where their entire existence has already passed us by - exploded into ‘life’, shone for millions of years then died and all the light from those events has long since passed the earth. But some of those early stars were so far away that it hasn’t reached us yet- as I say, the exact calculations are really complicated but let’s do some Simple stats.
The universe (we think) is roughly 13 billion years old, but it’s expanding at a fair chunk of the speed of light. So let’s say just for ease of numbers the furthest away/oldest thing that exists is currently 26 billion light years away. The furthest object we can detect defines what we call the observable universe, which is more or less a sphere centred on the earth. Once JW is in position, the size of the observable universe will be roughly 48 billion light years across (according to Wikipedia), so we will be able to see things 24 billion light years away or, put another way, from 24 billion years ago IN SPACETIME, which is flexy and not like the time on your watch - which is why there are things 24 billion light years away, even though the universe is 13 billion years old and nothing can move faster than the speed of light 😉 . At that point, the first stars would almost certainly have still been burning, so we will be able to detect them with JW.
These numbers being similar isn’t a coincidence though - JW was designed knowing looking much further back/away was kind of pointless given how we believe the universe was formed.
The interesting thing will be if we look at that early universe and don’t see what we expect to. Because that will tell us our theories need updating. As the saying goes, the most important exclamation in science is not ‘eureka’, it’s ‘that’s weird…’